Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue/20753/gracefulshutdown #22

Open
wants to merge 812 commits into
base: master
Choose a base branch
from

Conversation

labuladong
Copy link
Owner

Fixes #xyz

Master Issue: #xyz

PIP: #xyz

Motivation

Modifications

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@github-actions
Copy link

@labuladong Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

zymap and others added 27 commits July 18, 2023 11:43
…re conf (apache#20804)

### Motivation

The offload policies have limited the configurations for the offloaders.  That means if the offloader needs more configurations, we need to extend more fields in the OffloadPoliciesImpl. That doesn't make sense. We should make it extendable easily. Add a configuration map support to allow it to set more configurations.
…ocket.conf (apache#20840)

Motivation: Since the PR apache#16234 add the prop `cryptoKeyReaderFactoryClassName` for the WebSocket Proxy, but did not add this prop to `websocket.conf`.  This can make the script which try to replacement attribute a bit difficult to write

Modifications: add the conf `cryptoKeyReaderFactoryClassName` into the file `websocket.conf`
…ransactionBufferInternalStats API (apache#20330)

master apache#20291

### Motivation

Our primary goal is to improve the visibility and troubleshooting capabilities of the Pulsar Transaction Buffer by providing more detailed information about the snapshot stats and system topic internal status.
### Modifications

1. Enhance the existing TransactionBufferStats by adding information about snapshot stats, including the capital of the current segment, unseal aborted transaction ID size, and other related data. This will provide better visibility and troubleshooting capabilities for the Pulsar Transaction Buffer.
2. Introduce a new API for obtaining TransactionBufferInternalStats, allowing users to access the state of the system topic used for storing snapshots. This will facilitate problem investigation and resolution when issues arise with the transaction buffer.
…apache#20713)

Sasl authentication has two phases. In the first phase, the broker will temporarily cache the authentication info, and clean the cache at the end of the second phase.
When the client crashes after phase 1, or someone deliberately attacks, the auth info will always remain in the broker's memory.
If the client keeps reconnecting like this, the broker will soon OOM.
…e#20858)

### Motivation

Add `Audience.Private` annotation for topic compaction strategy. Since this interface only uses for load manager, other users should not use this interface.
…ndalone (apache#20709)

Fixes : apache#20617

### Motivation

Currently, clients' shutdown API does not behave consistently in sense that asynchronicity is not handled explicitly. So issue apache#20617 happens.

This PR will allow clients know that shutdown is actually triggered.

### Modifications

- Synchronize call to `POST /shutdown` on client side
- Asynchronize explicitly `pulsar().closeAsync()` invocation
…egistered if there has no message was sent (apache#20888)

Motivation: In the replication scenario, we want to produce messages on the native cluster and consume messages on the remote cluster, the producer and consumer both use a same schema, but the consumer cannot be registered if there has no messages in the topic yet.The root cause is that for the remote cluster, there is a producer who has been registered with `AUTO_PRODUCE_BYTES` schema, so there is no schema to check the compatibility.

Modifications: If there is no schema and only the replicator producer was registered, skip the compatibility check.
lhotari and others added 30 commits October 19, 2023 21:04
apache#21353)

### Motivation

See pip: apache#21129

### Modifications

Add  `BrokerTimeAverageDataResources`
…ation level (apache#21114)

### Motivation
This pip is to implement Read Committed and Read Uncommitted isolation levels for Pulsar transactions, allow consumers to configure isolation levels during the building process.
For more details, please refer to `pip-298.md`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.