Allow configuring replication schedule

[Shatur]

Right now we expose tick_schedule, which allows configuring the schedule on which the server tick is incremented. By default, it's FixedPostUpdate.

Incrementing the tick triggers replication in PostUpdate:

bevy_replicon/src/server.rs

Lines 158 to 166 in 60e3c5e

	PostUpdate,
	(
	prepare_messages,
	collect_mappings,
	collect_despawns,
	collect_removals,
	collect_changes,
	send_messages,
	)

However, if the fixed schedule runs multiple times, the tick will also be incremented multiple times (assuming it's incremented in the fixed schedule, which is the default). Replication, however, will happen only once. This could be problematic for prediction/interpolation because it's impossible to tell on which tick a value was changed.

We could allow configuring the replication schedule as well.

But maybe we shouldn't even separate replication it from the tick increment? I.e instead of running replication when the tick is incremented, we could include the tick increment inside the replication schedule itself. So it will be a single plugin setting that determines both when the tick is incremented and when replication is sent.

Thoughts?

[PROMETHIA-27]

imo:

• incrementing the tick must happen once per run of the simulation update loop (the actual game logic that then gets rendered in the rendering update loop). Typically the simulation update loop is the fixed update loop, because the simulation benefits from the determinism of a fixed update timestep, and the rendering update loop goes in the non-fixed update loop because we often want to configure the FPS independently of tick rate.
• doing replication should go in a non-fixed update loop, because we do not want to "pack" replication updates together; they should actually occur roughly equidistantly in real time. Thus we put them in the non-fixed update loop and maybe skip replication some frames to achieve a lower replication rate than frame rate, if desired. Achieving a higher replication rate than frame rate would require skipping rendering instead, since doing the fixed update schedule technique would just cluster replications together instead of making them occur at a faster equidistant rate.

So they should be separate, at least by default. To deal with not knowing what tick a value is from; we already have the information for this built into replicon. Every component is sent alongside a RepliconTick for its provenance. You can use this for prediction as long as you are able to cleanly map between replicon ticks and your game's concept of a tick. Otherwise you can attach a component storing your game's concept of a tick on each replicated entity, and because entities are updated atomically in replicon you can use that to determine the tick of the entity's replicated components.

Ofc you should be able to customize these things for various reasons (e.g. using a custom schedule instead of FixedMain specifically) but replicon is currently fine for the majority case I think.

[cBournhonesque]

Even if we schedule replication in FixedUpdate, it is still quite tricky to map between RepliconTick and FixedUpdate tick.

To contextualize, the issue we are trying to solve is that we need to know (for prediction/interpolation) the exact FixedUpdate tick that corresponds to a RepliconTick.

I see 3 main options:
A)

• provide a FixedUpdate tick in replicon (or a hook for users to manually update it) and use that everywhere. So if replication is done every 12 ticks, then two consecutive replication systems would use ReplicationTick=12 and then ReplicationTick=24.
  That way you don't need to worry about a mapping between RepliconTick and FixedUpdate tick.

Downsides:

• you cannot know exactly how many send 'schedules' were run between two ticks. I don't know if this assumption is used anywhere. Maybe in some sequential replication/buffering logic to ensure that we have received all messages (for UpdateMessages)?
• some features (ServerMutateTicks history) use the fact that RepliconTicks are consecutive

Pros:

• fairly easy

B)
Add the FixedUpdate tick in the packet, or provide a hook for the user to do so.
See: #707
This wouldn't waste extra space on UpdateMessages, but it might waste one extra byte on MutateMessages.
This is basically what i am doing now in lightyear

Downsides:

• 2/3 bytes waste on each packet
  Pros:
• easy
• proven to work

C)
Make sure that the mapping between RepliconTick and FixedUpdate tick is deterministic (for instance RepliconTick = FixedUpdateTick / 12) and possibly trigger the Replication-send systems inside FixedUpdate instead of PostUpdate. (what this issue is considering)

Downsides:

• huge waste if somehow replication runs multiple times in a frame (if we have 4 FixedUpdates in a frame and replication runs for two of them). In practice I think this is fine, since this feature would basically only be used if we want to replicate somewhat infrequently, for example at 10Hz or 20Hz, which is fairly standard.
• need the deterministic relationship between FixedUpdate tick and Replicon tick to always hold. I think this should be fine but it might cause some extra complexity if we have a server restart or something like that
• it seems kind of wasteful that we had access to data from a more recent tick but still decided to replicate data from an older tick. That means we might be receiving data from a slightly later tick than we should have, so it will hurt prediction/interpolation performance (interpolation will be slightly delayed, prediction will need to rollback for slightly more)

Pros:

• no waste of bandwidth

I think the 3rd downside of C) is kind of a deal-breaker.

[Shatur]

Edit: see the last message.

@PROMETHIA-27 but with what you're describing (which is basically the current behavior) you won't be able to reconstruct the component history for prediction/interpolation.
Let's consider this example:

• FixedUpdate 1, ServerTick increments to 1,
• FixedUpdate 2, ServerTick increments to 2,
• PostUpdate - Send values from FixedUpdate 2 and tick 2.

If a component was mutated on both schedules, you'll lose that value and tick 1 will be basically missed. I even expose things like need_history to let prediction/interpolation crates to see these values to in order to populate the history properly.

@cBournhonesque I think in A the 2 downside is a deal-breaker. I like the simplicity of B and I'll add a convenient API for you to do this, but let's discuss C first real quick. Based on what you and @PROMETHIA-27 said, I think I'm missing something. You mentioned this:

it seems kind of wasteful that we had access to data from a more recent tick but still decided to replicate data from an older tick.

But you'll get proper value history this way. I demonstrated this in the example above for @PROMETHIA-27. If we replicate once per multiple ticks - client may never see some values. Please, correct me if I'm wrong here 🙂

The only problem I see with C is that right now the replication is ACK-based, so if the component didn't change, we still send it twice (since we send since the last ack). But if we switch to NACK-based approach as we discussed in #697, the component will be included only in the first message, as it should.

[Shatur]

I think I get it now.
From the discussion in Discord with NiseVoid:

NiseVoid — 12:26 AM
I can see how this could be annoying with listen servers, I guess?
Shatur — 6:55 PM
That's my understanding, yeah. Because if we send replication only once per several fixed updates, client will never see all values, which affects interpolation/prediction, right?
I explained this in the issue.
NiseVoid — 7:01 PM
Yea, we'd be missing values, though at that point it's not too bad if the clients fill in the gap with their best gap
Shatur — 7:02 PM
So you would say it's not worth to waste bandwidth on old values?
Earlier we basically re-used what server already sent instead of discarding it (via things like need_history) 
NiseVoid — 7:05 PM
If the server already sent values, it's valuable for a client to store, but if the server has yet to send them but already has newer data it seems useless to send them
Plus it might indicate the server lagging so not adding extra load also seems preferable

So I think it's worth to just go with the option B as @cBournhonesque suggested. I'll just provide an API for it. @PROMETHIA-27 it could be useful for you too since you'll be able to attach the tick only once instead of duplicating it in each entity. Thoughts?

[cBournhonesque]

But you'll get proper value history this way. I demonstrated this in the example above for @PROMETHIA-27. If we replicate once per multiple ticks - client may never see some values. Please, correct me if I'm wrong here 🙂

That's true; in general I've been ok with it. When I send replication updates every 10 ticks I already accept that I am losing some fidelity by 'sampling' the state of the world more infrequently than at every tick. I'm making the choice of losing some accuracy in order to cut bandwidth costs (and some CPU costs)
I still use need_history=True because I don't want to lose even more precision on top of that. (i.e. if packets for RepliconTicks 1, 2, 3 were delayed and I first receive RepliconTick 4, I still want to add to my histories the RepliconTick 1,2,3 when I receive them) but I don't care about any updates that happened between consecutive RepliconTicks.

I'm not saying that this is the correct approach, your proposal to actually have the full exact history for every component sounds interesting!

I also think that the downside I presented for option C is a deal-breaker.
(if you have

FixedUpdate 1 - RepliconTick incremented - run replicaton-send systems
FixedUpdate 2 
PostUpdate

i.e. the packets you send are for FixedUpdate 1 even though you had a more recent state available)

[Shatur]

Yes, what you and @PROMETHIA-27 are describing makes total sense to me now. I think it's the best approach. Closing this issue in favor of #707?