I think this can't be implemented like this.

size_t PubSubClient::write(const uint8_t* buffer, size_t size) {
    const size_t rc = appendBuffer(buffer, size);
    if (rc != 0) {
        lastOutActivity = millis();
    }
    return rc;
}

We can't set lastOutActivity here, as appendBuffer() does not necessarily write to the the broker. And we need this to ping the broker in time ...

But setting lastOutActivity in flushBuffer() only, also won't work in all cases (e.g. if the application takes to long to fill all the needed data), as here

To your writeNextMsgId() concern. I don't think there is a problem. writeNextMsgId() is used in endPublish() there you did the flushBuffer() before and in subscribe() / unsubscribe() - don't care ...

Well since appendBuffer may call flushBuffer, which does actual communications (and thus should set the lastOutActivity), I agree that appendBuffer itself and thus also these write calls should not set this lastOutActivity .

I was thinking... maybe all _client.write calls should use this appendBuffer, just to make sure there isn't any risk of mixing up different calls.

This was my first thought too. But then more things need to be rewritten ...

And yes, at least here

@TD-er Did some implementations mentioned above. Looks quite promising. If you have time you may have a look over it.

@TD-er We could merge appendBuffer(uint8_t data)

This PR fixes #29.

This fixes #36

@TD-er If you could take a look over it, I'd like to merge this. Tested it for a few weeks on a device of mine (but only uses publish_P()).

@TD-er Is there any chance to finish this PR from your side? I would love to implement some other open issues but this might make extra work merging here ...

I will have a look

OK, I loaded the files into my (ESPEasy) project and it builds just fine, so that's a plus ;)

Also normal MQTT operations do seem to work just fine.

Right now I don't have the setup here to dump large chunks, so that's a bit hard to test right now.

Just as a cosmetic remark... There is no consistency in naming of member variables.
I would propose to start all members with an _

I don't see any obvious mistakes, so LGTM

Just as a cosmetic remark... There is no consistency in naming of member variables.
I would propose to start all members with an _

Yeah, that's what I want to do next ... see #60.

Hm, yes. Testing with big data is a good point. Maybe I can create a unit test for that ...

Unit test runs green

• publishes with long payload message (> buffer size) ✓

With a simple code I just send 6k payload data ...

/**
 * @brief Publish a file from LittleFS to MQTT topic using beginPublish/write/endPublish
 */
void mqttPublishFile(const char *lfs_path, const char *mqtt_topic) {
  if (!LittleFS.exists(lfs_path)) {
    ERRORF_P("%s: LittleFS file not found: %s" LF, __func__, lfs_path);
    return;
  }
  File f = LittleFS.open(lfs_path, "r");
  if (!f) {
    ERRORF_P("%s: Failed to open file: %s" LF, __func__, lfs_path);
    return;
  }

  size_t remaining = f.size();
  INFOF_P("%s: Publishing file %s (%u bytes) to topic %s" LF, __func__, lfs_path, (unsigned)remaining, mqtt_topic);

  if (!m_mqtt_client.beginPublish(mqtt_topic, remaining, false)) {
    ERRORF_P("%s: beginPublish failed for topic %s" LF, __func__, mqtt_topic);
    f.close();
    return;
  }

  const size_t BUF_SZ = 512;
  uint8_t buf[BUF_SZ];
  while (remaining > 0) {
    size_t toRead = (remaining > BUF_SZ) ? BUF_SZ : remaining;
    int r = f.read(buf, toRead);
    if (r <= 0) break;
    m_mqtt_client.write(buf, r);
    remaining -= r;
  }

  m_mqtt_client.endPublish();
  f.close();
}

    // publish LittleFS file "/index..html" to topic "test"
    mqttPublishFile("/index.html", "test");

mqttPublishFile: Publishing file /index.html (6625 bytes) to topic test

So everything look good for me as well. Merging that.