Recovering buffered SD data with authenticated HTTP POST
Uploading SD files over HTTP POST closes the loop of an offline-first datalogger: buffering data during an outage is only useful if you can get it back without walking to the machine. This example streams a daily JSON file from the ESP32 PLC's SD card to a Node-RED endpoint with basic authentication — the recovery path used in a real textile machine monitoring deployment whenever the network drops and returns.
Streaming straight from the card
The key line is http.sendRequest("POST", &file, file.size()) — HTTPClient takes the open File object and streams it chunk by chunk, so the ESP32 never has to load the file into RAM. A multi-megabyte day of machine logs uploads as easily as a tiny one, with the same few kilobytes of memory in play. Content-Type text/plain and a generous 15-second timeout accommodate the larger transfers, and the file handle is closed immediately afterwards whether the request succeeded or not.
Auth and a filename header
setAuthorization(HTTP_USER, HTTP_PASS) adds HTTP basic authentication, so the recovery endpoint is not an open door for anything else living on the plant network. A custom X-File-Name header travels with the request and tells the server exactly which daily file it is receiving, without having to parse anything out of the body. On the receiving end, a Node-RED HTTP-in node verifies the credentials, sanitises that filename and writes the payload into a per-plant recovery folder on disk.
Only delete after a 200
The golden rule of recovery: the PLC only moves a file to the /enviados folder when the server answered HTTP 200. Any other code — timeout, 401, 500 — leaves the file untouched on the card for the next attempt. In production the upload is triggered remotely by an MQTT command, so the whole recovery is done from a desk.
A snippet from the implementation
Straight from the example as deployed on the ESP32 PLC — copy it freely:
void setup() {
Serial.begin(115200);
SD.begin();
WiFi.mode(WIFI_STA);
WiFi.begin(WIFI_SSID, WIFI_PASS);
uint32_t t0 = millis();
while (WiFi.status() != WL_CONNECTED && millis() - t0 < 15000) delay(500);
Serial.println("Ready. Type the name of the file to upload,");
Serial.println("e.g.: /2026-06-10.json");
}
The full example is a complete program — wiring header, setup and main loop — ready to adapt to your application.
Frequently asked questions
Why HTTP POST instead of replaying the data over MQTT?
A day of logs can be thousands of messages; replaying them through a broker is slow and floods subscribers. One authenticated POST moves the whole file in a single transaction with a clear success signal.
How is the upload triggered after an outage?
In the production system an MQTT command like "upload 2026-06-10.json" arrives on the PLC's command topic. You can also automate it — on reconnection, walk the SD inventory and push every file not yet marked as sent.
Is basic authentication enough security?
On an isolated plant VLAN it is a reasonable baseline that keeps casual access out. If the traffic crosses less trusted networks, put the endpoint behind a reverse proxy with TLS and stronger credentials.