Test all the things. Here's a WIP CHANGELOG:

Routing:
  * Unicode characters are accepted anywhere in route paths. (#998)
  * Dyanmic query values can (and must) be any `FromForm` type. The `Form` type
    is no longer useable in any query parameter type.

Capped
  * A new `Capped` type is used to indicate when data has been truncated due to
    incoming data limits. It allows checking whether data is complete or
    truncated. `DataStream` methods returns `Capped` types.
  * Several `Capped<T>` types implement `FromData`, `FromForm`.
  * HTTP 413 (Payload Too Large) errors are now returned when the data limit is
    exceeded. (resolves #972)

Hierarchical Limits
  * Data limits are now hierarchical, delimited with `/`. A limit of `a/b/c`
    falls back to `a/b` then `a` when not set.

Temporary Files
  * A new `TempFile` data and form guard allows streaming data directly to a
    file which can then be persisted.
  * A new `temp_dir` config parameter specifies where to store `TempFile`.
  * The limits `file` and `file/$ext`, where `$ext` is the file extension,
    determines the data limit for a `TempFile`.

Forms Revamp
  * All form related types now reside in a new `form` module.
  * Multipart forms are supported. (resolves #106)
  * Collections are supported in body forms and queries. (resolves #205)
  * Nested forms and structures are supported. (resolves #313)
  * Form fields can be ad-hoc validated with `#[field(value = expr)]`.

Core:
  * `&RawStr` no longer implements `FromParam`.
  * `&str` implements `FromParam`, `FromData`, `FromForm`.
  * `FromTransformedData` was removed.
  * `FromData` gained a lifetime for use with request-local data.
  * All dynamic paramters in a query string must typecheck as `FromForm`.
  * `FromFormValue` removed in favor of `FromFormField`.
  * Dyanmic paramters, form values are always percent-decoded.
  * The default error HTML is more compact.
  * `&Config` is a request guard.
  * The `DataStream` interface was entirely revamped.
  * `State` is only exported via `rocket::State`.
  * A `request::local_cache!()` macro was added for storing values in
    request-local cache without consideration for type uniqueness by using a
    locally generated anonymous type.
  * `Request::get_param()` is now `Request::param()`.
  * `Request::get_segments()` is now `Request::segments()`, takes a range.
  * `Request::get_query_value()` is now `Request::query_value()`, can parse any
    `FromForm` including sequences.
  * `std::io::Error` implements `Responder` as `Debug<std::io::Error>`.
  * `(Status, R)` where `R: Responder` implements `Responder` by setting
    overriding the `Status` of `R`.
  * The name of a route is printed first during route matching.
  * `FlashMessage` now only has one lifetime generic.

HTTP:
  * `RawStr` implements `serde::{Serialize, Deserialize}`.
  * `RawStr` implements _many_ more methods, in particular, those related to the
    `Pattern` API.
  * `RawStr::from_str()` is now `RawStr::new()`.
  * `RawStr::url_decode()` and `RawStr::url_decode_lossy()` only allocate as
    necessary, return `Cow`.
  * `(Status, R)` where `R: Responder` is a responder that overwrites the status
    of `R` to `Status`.
  * `Status` implements `Default` with `Status::Ok`.
  * `Status` implements `PartialEq`, `Eq`, `Hash`, `PartialOrd`, `Ord`.
  * Authority and origin part of `Absolute` can be modified with new
    `Absolute::{with,set}_authority()`, `Absolute::{with,set}_origin()` methods.
  * `Origin::segments()` was removed in favor of methods split into query and
    path parts and into raw and decoded parts.
  * The `Segments` iterator is signficantly smarter. Returns `&str`.
  * `Segments::into_path_buf()` is now `Segments::to_path_buf()`, doesn't
    consume.
  * A new `QuerySegments` is the analogous query segment iterator.
  * Once set, the `expires` field on private cookies is not overwritten.
    (resolves #1506)
  * `Origin::path()` and `Origin::query()` return `&RawStr`, not `&str`.

Codegen:
  * Preserve more spans in `uri!` macro.
  * `FromFormValue` derive removed; `FromFormField` added.
  * The `form` `FromForm` and `FromFormField` field attribute is now named
    `field`. `#[form(field = ..)]` is now `#[form(name = ..)]`.

Examples:
  * `form_validation` and `form_kitchen_sink` removed in favor of `forms`
  * `rocket_contrib::Json` implements `FromForm`.
  * The `json!` macro is exported as `rocket_contrib::json::json`.
  * `rocket_contrib::MsgPack` implements `FromForm`.
  * Added clarifying docs to `StaticFiles`.
  * The `hello_world` example uses unicode in paths.

Internal:
  * Codegen uses new `exports` module with the following conventions:
    - Locals starts with `__` and are lowercased.
    - Rocket modules start with `_` are are lowercased.
    - Stdlib types start with `_` are are titlecased.
    - Rocket types are titlecased.
  * A `header` module was added to `http`, contains header types.
  * `SAFETY` is used as doc-string keyword for `unsafe` related comments.
  * The `Uri` parser no longer recognizes Rocket route URIs.

Author: SergioBenitez

contrib/lib/src/json.rs

@@ -34,18 +34,21 @@ pub use serde_json::{json_internal, json_internal_vec};
 ///
 /// ## Receiving JSON
 ///
-/// If you're receiving JSON data, simply add a `data` parameter to your route
-/// arguments and ensure the type of the parameter is a `Json<T>`, where `T` is
-/// some type you'd like to parse from JSON. `T` must implement [`Deserialize`]
-/// from [`serde`]. The data is parsed from the HTTP request body.
+/// `Json` is both a data guard and a form guard.
+///
+/// ### Data Guard
+///
+/// To parse request body data as JSON , add a `data` route argument with a
+/// target type of `Json<T>`, where `T` is some type you'd like to parse from
+/// JSON. `T` must implement [`serde::Deserialize`].
 ///
 /// ```rust
 /// # #[macro_use] extern crate rocket;
 /// # extern crate rocket_contrib;
 /// # type User = usize;
 /// use rocket_contrib::json::Json;
 ///
-/// #[post("/users", format = "json", data = "<user>")]
+/// #[post("/user", format = "json", data = "<user>")]
 /// fn new_user(user: Json<User>) {
 ///     /* ... */
 /// }
@@ -56,6 +59,30 @@ pub use serde_json::{json_internal, json_internal_vec};
 /// "application/json" as its `Content-Type` header value will not be routed to
 /// the handler.
 ///
+/// ### Form Guard
+///
+/// `Json<T>`, as a form guard, accepts value and data fields and parses the
+/// data as a `T`. Simple use `Json<T>`:
+///
+/// ```rust
+/// # #[macro_use] extern crate rocket;
+/// # extern crate rocket_contrib;
+/// # type Metadata = usize;
+/// use rocket::form::{Form, FromForm};
+/// use rocket_contrib::json::Json;
+///
+/// #[derive(FromForm)]
+/// struct User<'r> {
+///     name: &'r str,
+///     metadata: Json<Metadata>
+/// }
+///
+/// #[post("/user", data = "<form>")]
+/// fn new_user(form: Form<User<'_>>) {
+///     /* ... */
+/// }
+/// ```
+///
 /// ## Sending JSON
 ///
 /// If you're responding with JSON data, return a `Json<T>` type, where `T`

contrib/lib/src/msgpack.rs

@@ -34,11 +34,13 @@ pub use rmp_serde::decode::Error;
 ///
 /// ## Receiving MessagePack
 ///
-/// If you're receiving MessagePack data, simply add a `data` parameter to your
-/// route arguments and ensure the type of the parameter is a `MsgPack<T>`,
-/// where `T` is some type you'd like to parse from MessagePack. `T` must
-/// implement [`Deserialize`] from [`serde`]. The data is parsed from the HTTP
-/// request body.
+/// `MsgPack` is both a data guard and a form guard.
+///
+/// ### Data Guard
+///
+/// To parse request body data as MessagePack , add a `data` route argument with
+/// a target type of `MsgPack<T>`, where `T` is some type you'd like to parse
+/// from JSON. `T` must implement [`serde::Deserialize`].
 ///
 /// ```rust
 /// # #[macro_use] extern crate rocket;
@@ -57,6 +59,30 @@ pub use rmp_serde::decode::Error;
 /// "application/msgpack" as its first `Content-Type:` header parameter will not
 /// be routed to this handler.
 ///
+/// ### Form Guard
+///
+/// `MsgPack<T>`, as a form guard, accepts value and data fields and parses the
+/// data as a `T`. Simple use `MsgPack<T>`:
+///
+/// ```rust
+/// # #[macro_use] extern crate rocket;
+/// # extern crate rocket_contrib;
+/// # type Metadata = usize;
+/// use rocket::form::{Form, FromForm};
+/// use rocket_contrib::msgpack::MsgPack;
+///
+/// #[derive(FromForm)]
+/// struct User<'r> {
+///     name: &'r str,
+///     metadata: MsgPack<Metadata>
+/// }
+///
+/// #[post("/users", data = "<form>")]
+/// fn new_user(form: Form<User<'_>>) {
+///     /* ... */
+/// }
+/// ```
+///
 /// ## Sending MessagePack
 ///
 /// If you're responding with MessagePack data, return a `MsgPack<T>` type,

contrib/lib/src/uuid.rs

@@ -21,7 +21,7 @@ use std::str::FromStr;
 use std::ops::Deref;
 
 use rocket::request::FromParam;
-use rocket::form::{self, FromFormField, Errors, ValueField};
+use rocket::form::{self, FromFormField, ValueField};
 
 type ParseError = <self::uuid_crate::Uuid as FromStr>::Err;
 
@@ -110,7 +110,7 @@ impl<'a> FromParam<'a> for Uuid {
 }
 
 impl<'v> FromFormField<'v> for Uuid {
-    fn from_value(field: ValueField<'v>) -> Result<Self, Errors<'v>> {
+    fn from_value(field: ValueField<'v>) -> form::Result<'v, Self> {
         Ok(field.value.parse().map_err(form::error::Error::custom)?)
     }
 }

core/codegen/src/attribute/route.rs

@@ -194,16 +194,27 @@ fn data_expr(ident: &syn::Ident, ty: &syn::Type) -> TokenStream {
 }
 
 fn query_exprs(route: &Route) -> Option<TokenStream> {
-    use devise::ext::Split6;
+    use devise::ext::{Split2, Split6};
 
     define_spanned_export!(Span::call_site() =>
         __req, __data, _log, _form, Outcome, _Ok, _Err, _Some, _None
     );
 
     let query_segments = route.attribute.path.query.as_ref()?;
 
-    // NOTE: We only care about dynamic parameters since the router will only
-    // send us request where the static parameters match.
+    // Record all of the static parameters for later filtering.
+    let (raw_name, raw_value) = query_segments.iter()
+        .filter(|s| !s.is_dynamic())
+        .map(|s| {
+            let name = s.name.name();
+            match name.find('=') {
+                Some(i) => (&name[..i], &name[i + 1..]),
+                None => (name, "")
+            }
+        })
+        .split2();
+
+    // Now record all of the dynamic parameters.
     let (name, matcher, ident, init_expr, push_expr, finalize_expr) = query_segments.iter()
         .filter(|s| s.is_dynamic())
         .map(|s| (s, s.name.name(), route.find_input(&s.name).expect("dynamic has input")))
@@ -235,9 +246,12 @@ fn query_exprs(route: &Route) -> Option<TokenStream> {
         #(let mut #ident = #init_expr;)*
 
         for _f in #__req.query_fields() {
-            match _f.name.key_lossy().as_str() {
-                // FIXME: Need to skip raw so we don't push into trailing.
-                #(#matcher => #push_expr,)*
+            let _raw = (_f.name.source().as_str(), _f.value);
+            let _key = _f.name.key_lossy().as_str();
+            match (_raw, _key) {
+                // Skip static parameters so <param..> doesn't see them.
+                #(((#raw_name, #raw_value), _) => { /* skip */ },)*
+                #((_, #matcher) => #push_expr,)*
                 _ => { /* in case we have no trailing, ignore all else */ },
             }
         }

core/codegen/tests/route-data.rs

@@ -9,26 +9,26 @@ use rocket::form::Form;
 // Test that the data parameters works as expected.
 
 #[derive(FromForm)]
-struct Inner {
-    field: String
+struct Inner<'r> {
+    field: &'r str
 }
 
-struct Simple(String);
+struct Simple<'r>(&'r str);
 
 #[async_trait]
-impl<'r> FromData<'r> for Simple {
+impl<'r> FromData<'r> for Simple<'r> {
     type Error = std::io::Error;
 
     async fn from_data(req: &'r Request<'_>, data: Data) -> data::Outcome<Self, Self::Error> {
-        String::from_data(req, data).await.map(Simple)
+        <&'r str>::from_data(req, data).await.map(Simple)
     }
 }
 
 #[post("/f", data = "<form>")]
-fn form(form: Form<Inner>) -> String { form.into_inner().field }
+fn form<'r>(form: Form<Inner<'r>>) -> &'r str { form.into_inner().field }
 
 #[post("/s", data = "<simple>")]
-fn simple(simple: Simple) -> String { simple.0 }
+fn simple<'r>(simple: Simple<'r>) -> &'r str { simple.0 }
 
 #[test]
 fn test_data() {

core/codegen/tests/route.rs

@@ -148,3 +148,151 @@ mod scopes {
         rocket::ignite().mount("/", rocket::routes![hello, world])
     }
 }
+
+use rocket::form::Contextual;
+
+#[derive(Default, Debug, PartialEq, FromForm)]
+struct Filtered<'r> {
+    bird: Option<&'r str>,
+    color: Option<&'r str>,
+    cat: Option<&'r str>,
+    rest: Option<&'r str>,
+}
+
+#[get("/?bird=1&color=blue&<bird>&<color>&cat=bob&<rest..>")]
+fn filtered_raw_query(bird: usize, color: &str, rest: Contextual<'_, Filtered<'_>>) -> String {
+    assert_ne!(bird, 1);
+    assert_ne!(color, "blue");
+    assert_eq!(rest.value.unwrap(), Filtered::default());
+
+    format!("{} - {}", bird, color)
+}
+
+#[test]
+fn test_filtered_raw_query() {
+    let rocket = rocket::ignite().mount("/", routes![filtered_raw_query]);
+    let client = Client::untracked(rocket).unwrap();
+
+    #[track_caller]
+    fn run(client: &Client, birds: &[&str], colors: &[&str], cats: &[&str]) -> (Status, String) {
+        let join = |slice: &[&str], name: &str| slice.iter()
+            .map(|v| format!("{}={}", name, v))
+            .collect::<Vec<_>>()
+            .join("&");
+
+        let q = format!("{}&{}&{}",
+            join(birds, "bird"),
+            join(colors, "color"),
+            join(cats, "cat"));
+
+        let response = client.get(format!("/?{}", q)).dispatch();
+        let status = response.status();
+        let body = response.into_string().unwrap();
+
+        (status, body)
+    }
+
+    let birds = &["2", "3"];
+    let colors = &["red", "blue", "green"];
+    let cats = &["bob", "bob"];
+    assert_eq!(run(&client, birds, colors, cats).0, Status::NotFound);
+
+    let birds = &["2", "1", "3"];
+    let colors = &["red", "green"];
+    let cats = &["bob", "bob"];
+    assert_eq!(run(&client, birds, colors, cats).0, Status::NotFound);
+
+    let birds = &["2", "1", "3"];
+    let colors = &["red", "blue", "green"];
+    let cats = &[];
+    assert_eq!(run(&client, birds, colors, cats).0, Status::NotFound);
+
+    let birds = &["2", "1", "3"];
+    let colors = &["red", "blue", "green"];
+    let cats = &["bob", "bob"];
+    assert_eq!(run(&client, birds, colors, cats).1, "2 - red");
+
+    let birds = &["1", "2", "1", "3"];
+    let colors = &["blue", "red", "blue", "green"];
+    let cats = &["bob"];
+    assert_eq!(run(&client, birds, colors, cats).1, "2 - red");
+
+    let birds = &["5", "1"];
+    let colors = &["blue", "orange", "red", "blue", "green"];
+    let cats = &["bob"];
+    assert_eq!(run(&client, birds, colors, cats).1, "5 - orange");
+}
+
+#[derive(Debug, PartialEq, FromForm)]
+struct Dog<'r> {
+    name: &'r str,
+    age: usize
+}
+
+#[derive(Debug, PartialEq, FromForm)]
+struct Q<'r> {
+    dog: Dog<'r>
+}
+
+#[get("/?<color>&color=red&<q..>")]
+fn query_collection(color: Vec<&str>, q: Q<'_>) -> String {
+    format!("{} - {} - {}", color.join("&"), q.dog.name, q.dog.age)
+}
+
+#[get("/?<color>&color=red&<dog>")]
+fn query_collection_2(color: Vec<&str>, dog: Dog<'_>) -> String {
+    format!("{} - {} - {}", color.join("&"), dog.name, dog.age)
+}
+
+#[test]
+fn test_query_collection() {
+    #[track_caller]
+    fn run(client: &Client, colors: &[&str], dog: &[&str]) -> (Status, String) {
+        let join = |slice: &[&str], prefix: &str| slice.iter()
+            .map(|v| format!("{}{}", prefix, v))
+            .collect::<Vec<_>>()
+            .join("&");
+
+        let q = format!("{}&{}", join(colors, "color="), join(dog, "dog."));
+        let response = client.get(format!("/?{}", q)).dispatch();
+        (response.status(), response.into_string().unwrap())
+    }
+
+    fn run_tests(rocket: rocket::Rocket) {
+        let client = Client::untracked(rocket).unwrap();
+
+        let colors = &["blue", "green"];
+        let dog = &["name=Fido", "age=10"];
+        assert_eq!(run(&client, colors, dog).0, Status::NotFound);
+
+        let colors = &["red"];
+        let dog = &["name=Fido"];
+        assert_eq!(run(&client, colors, dog).0, Status::NotFound);
+
+        let colors = &["red"];
+        let dog = &["name=Fido", "age=2"];
+        assert_eq!(run(&client, colors, dog).1, " - Fido - 2");
+
+        let colors = &["red", "blue", "green"];
+        let dog = &["name=Fido", "age=10"];
+        assert_eq!(run(&client, colors, dog).1, "blue&green - Fido - 10");
+
+        let colors = &["red", "blue", "green"];
+        let dog = &["name=Fido", "age=10", "toy=yes"];
+        assert_eq!(run(&client, colors, dog).1, "blue&green - Fido - 10");
+
+        let colors = &["blue", "red", "blue"];
+        let dog = &["name=Fido", "age=10"];
+        assert_eq!(run(&client, colors, dog).1, "blue&blue - Fido - 10");
+
+        let colors = &["blue", "green", "red", "blue"];
+        let dog = &["name=Max+Fido", "age=10"];
+        assert_eq!(run(&client, colors, dog).1, "blue&green&blue - Max Fido - 10");
+    }
+
+    let rocket = rocket::ignite().mount("/", routes![query_collection]);
+    run_tests(rocket);
+
+    let rocket = rocket::ignite().mount("/", routes![query_collection_2]);
+    run_tests(rocket);
+}