Recovering buffered SD data with authenticated HTTP POST
Uploading SD files over HTTP POST closes the loop of an offline-first datalogger: buffering data during an outage is only useful if you can get it back without walking to the machine. This example streams a daily JSON file from the ESP32 PLC's SD card to a Node-RED endpoint with basic authentication, the recovery path used in a real textile machine monitoring deployment whenever the network drops and returns.
Streaming straight from the card
The key line is http.sendRequest("POST", &file, file.size()): HTTPClient takes the open File object and streams it chunk by chunk, so the ESP32 never has to load the file into RAM. A multi-megabyte day of machine logs uploads as easily as a tiny one, with the same few kilobytes of memory in play. Content-Type text/plain and a generous 15-second timeout accommodate the larger transfers, and the file handle is closed immediately afterwards whether the request succeeded or not.
Auth and a filename header
setAuthorization(HTTP_USER, HTTP_PASS) adds HTTP basic authentication, so the recovery endpoint is not an open door for anything else living on the plant network. A custom X-File-Name header travels with the request and tells the server exactly which daily file it is receiving, without having to parse anything out of the body. On the receiving end, a Node-RED HTTP-in node verifies the credentials, sanitises that filename and writes the payload into a per-plant recovery folder on disk.
Only delete after a 200
The golden rule of recovery: the PLC only moves a file to the /enviados folder when the server answered HTTP 200. Any other code, timeout, 401, 500, leaves the file untouched on the card for the next attempt. In production the upload is triggered remotely by an MQTT command, so the whole recovery is done from a desk.
A snippet from the implementation
Straight from the example as deployed on the ESP32 PLC: copy it freely:
void setup() {
Serial.begin(115200);
SD.begin();
WiFi.mode(WIFI_STA);
WiFi.begin(WIFI_SSID, WIFI_PASS);
uint32_t t0 = millis();
while (WiFi.status() != WL_CONNECTED && millis() - t0 < 15000) delay(500);
Serial.println("Ready. Type the name of the file to upload,");
Serial.println("e.g.: /2026-06-10.json");
}
The full example is a complete program, wiring header, setup and main loop, ready to adapt to your application.
Frequently asked questions
Why HTTP POST instead of replaying the data over MQTT?
A day of logs can be thousands of messages; replaying them through a broker is slow and floods subscribers. One authenticated POST moves the whole file in a single transaction with a clear success signal.
How is the upload triggered after an outage?
In the production system an MQTT command like "upload 2026-06-10.json" arrives on the PLC's command topic. You can also automate it: on reconnection, walk the SD inventory and push every file not yet marked as sent.
Is basic authentication enough security?
On an isolated plant VLAN it is a reasonable baseline that keeps casual access out. If the traffic crosses less trusted networks, put the endpoint behind a reverse proxy with TLS and stronger credentials.