`#[serde(flatten)]` breaks deserialization

[kyle-mccarthy]

Describe the bug

If a struct contains a field annotated with #[serde(flatten)], it cannot be deserialized by sonic.

To Reproduce

Cargo.toml

[package]
name = "repro"
version = "0.1.0"
edition = "2021"


[dependencies]
sonic-rs = "0.3.13"
serde = { version = "1.0", features = ["derive"] }

main.rs

fn main() {
    #[derive(serde::Deserialize)]
    struct Repro {
        expected_field: i32,
        #[serde(flatten)]
        others: std::collections::HashMap<String, sonic_rs::Value>,
    }

    let json = r#"
    {
        "expected_field": 1,
        "unexpected_field": 2
    }
    "#;

    let _repro: Repro = sonic_rs::from_str(json).unwrap();
}

Expected behavior

Deserialization to succeed and for others to contain the "unexpected_field".

Actual behavior

thread 'main' panicked at src/main.rs:16:50:
called Result::unwrap() on an Err value: invalid type: newtype struct, expected a valid json at line 5 column 4

sonic-rs version:

latest (0.3.13)

Environment:

macos

[liuq19]

thanks, we will investigate that

[liuq19]

the problem is similar as serde-rs/serde#1183 and serde-rs/json#1099

[tmuzy]

Hi, I encountered this issue as well, except that instead of throwing an error, the resulting sonic_rs::Value does not contain the flatten derived fields.

Is that related or should I open a separate issue ? 😃

[liuq19]

The problem is becuase serde-rs/serde#1183 and we will wait the PR serde-rs/serde#2912

[countradooku]

Any workaround ?

[liuq19]

could implement the customed Deserialize for Repo.

use sonic_rs;
use serde::de::{self, Deserializer, MapAccess, Visitor};
use serde::Deserialize;
use std::{collections::HashMap, fmt};

#[derive(Debug)]
struct Repro {
    expected_field: i32,
    others: HashMap<String, sonic_rs::Value>,
}

impl<'de> Deserialize<'de> for Repro {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        struct ReproVisitor;

        impl<'de> Visitor<'de> for ReproVisitor {
            type Value = Repro;

            fn expecting(&self, f: &mut fmt::Formatter) -> fmt::Result {
                f.write_str("expect a json object")
            }

            fn visit_map<M>(self, mut map: M) -> Result<Self::Value, M::Error>
            where
                M: MapAccess<'de>,
            {
                let mut expected_field = None;
                let mut others = HashMap::new();

                while let Some(key) = map.next_key::<String>()? {
                    if key == "expected_field" {
                        if expected_field.replace(map.next_value::<i32>()?).is_some() {
                            return Err(de::Error::duplicate_field("expected_field"));
                        }
                    } else {
                        let value: sonic_rs::Value = map.next_value()?;
                        others.insert(key, value);
                    }
                }

                let expected_field =
                    expected_field.ok_or_else(|| de::Error::missing_field("expected_field"))?;
                Ok(Repro { expected_field, others })
            }
        }

        deserializer.deserialize_map(ReproVisitor)
    }
}

fn main()  {
    let json = r#"
    {
        "expected_field": 1,
        "unexpected_field": 2,
        "another": { "nested": true }
    }"#;

    let parsed: Repro = sonic_rs::from_str(json).unwrap();
    println!("{parsed:#?}");
}