The good news is I narrowed it down.
The "ANYSAFE" auth method can cause a form to be reposted from scratch.
In this situation, the user-provided readfunction callback is switched to the standard fread callback, and then called with the user-provided readdata pointer, which the fread callback is incorrectly casting and writing to.
Valgrind also has a heap of warnings.
First, is the gdb output showing when the fread_func is reset, and then called.
Note: readstream is my user callback function.
Curl_FormReader is the built-in callback function
Curl_FormReader casts my user callback data 0x121 and then tries to access it.
Then, the code.
It has lots of odd variables and padding, I used them to force a segfault.
I believe your compiler may not crash in the same way, so I'm attaching a lot of output.
Then, GDB and Valgrind output
Breakpoint 6, Curl_http (conn=0x615e20, done=0x7fffffffdeb3) at /project/curl/lib/http.c:2454
2454 http->form.fread_func = data->set.fread_func;
(gdb) p data->set.fread_func
$15 = (curl_read_callback) 0x40112c <readcallback(char*, size_t, size_t, void*)>
(gdb) c
Continuing.
Breakpoint 6, Curl_http (conn=0x615e20, done=0x7fffffffdeb3) at /project/curl/lib/http.c:2454
2454 http->form.fread_func = data->set.fread_func;
(gdb) p data->set.fread_func
$16 = (curl_read_callback) 0x7ffff78460fe <Curl_FormReader>
(gdb) c
Continuing.
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7846138 in Curl_FormReader (
buffer=0x608689 '-' <repeats 26 times>, "fdf41c87d3c77191\r\nContent-Disposition: form-data; name=\"file\"; filename=\"filename\"\r\n\r\n", size=1, nitems=16384, mydata=0x121)
at /project/curl/lib/formdata.c:1453
1453 if(!form->data)
(gdb) p form
$17 = (struct Form *) 0x121
(gdb) The code:
#include <curl/curl.h>
#include <string>
using std::string;
const char* BASE_URL2 = "http://mysite?action=put";
const string BASE_URL = "http://mysite?action=put";
const string SOMETHING = "1";
static size_t readcallback(char *buffer, size_t size, size_t nitems, void *instream)
{
return 0;
}
struct SomeStruct
{
SomeStruct( string const& url )
{
}
};
int main(int argc, char **argv)
{
struct curl_httppost *formpost = NULL;
struct curl_httppost *lastptr = NULL;
CURL * handle = NULL;
curl_slist * headerlist = NULL;
const char* fn = "filename";
char space2[17]; // 17 is the sweet spot
char space[8+8+8];
SomeStruct curl(BASE_URL2);
curl_global_init(CURL_GLOBAL_ALL);
handle = (curl_easy_init());
headerlist = (curl_slist_append(NULL,"Expect:"));
// we don't want "Expect: 100-continue" -- it screws up proxy
curl_easy_setopt(handle, CURLOPT_HTTPHEADER, headerlist);
curl_easy_setopt(handle, CURLOPT_URL, BASE_URL2);
curl_easy_setopt(handle, CURLOPT_READFUNCTION, readcallback);
// authentication (at the server end)
curl_easy_setopt(handle, CURLOPT_HTTPAUTH, CURLAUTH_ANYSAFE);
curl_easy_setopt(handle, CURLOPT_USERNAME, "");
curl_easy_setopt(handle, CURLOPT_PASSWORD, "");
curl_formadd( &formpost, &lastptr,
CURLFORM_COPYNAME, "file",
CURLFORM_STREAM, (void*)0x121,
CURLFORM_CONTENTSLENGTH, 0,//reader_data.size,
CURLFORM_FILENAME, fn,
CURLFORM_END
);
curl_easy_setopt(handle, CURLOPT_HTTPPOST, formpost);
curl_easy_perform(handle);
// should die by now
return 0;
}The GDB output:
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7846138 in Curl_FormReader (
buffer=0x608689 '-' <repeats 26 times>, "4b8b37d30d5acaa0\r\nContent-Disposition: form-data; name=\"file\"; filename=\"filename\"\r\n\r\n", size=1, nitems=16384, mydata=0x121)
at /project/curl/lib/formdata.c:1453
1453 if(!form->data)
(gdb) bt
#0 0x00007ffff7846138 in Curl_FormReader (
buffer=0x608689 '-' <repeats 26 times>, "4b8b37d30d5acaa0\r\nContent-Disposition: form-data; name=\"file\"; filename=\"filename\"\r\n\r\n", size=1, nitems=16384, mydata=0x121)
at /project/curl/lib/formdata.c:1453
#1 0x00007ffff784604b in readfromfile (form=0x616838,
buffer=0x608689 '-' <repeats 26 times>, "4b8b37d30d5acaa0\r\nContent-Disposition: form-data; name=\"file\"; filename=\"filename\"\r\n\r\n", size=16384) at /project/curl/lib/formdata.c:1413
#2 0x00007ffff784617f in Curl_FormReader (
buffer=0x608689 '-' <repeats 26 times>, "4b8b37d30d5acaa0\r\nContent-Disposition: form-data; name=\"file\"; filename=\"filename\"\r\n\r\n", size=1, nitems=16384, mydata=0x616838)
at /project/curl/lib/formdata.c:1458
#3 0x00007ffff786ed6d in Curl_fillreadbuffer (conn=0x615e20, bytes=16384, nreadp=0x7fffffffdda4)
at /project/curl/lib/transfer.c:118
#4 0x00007ffff7870168 in readwrite_upload (data=0x603d80, conn=0x615e20, k=0x603df8, didwhat=0x7fffffffde2c)
at /project/curl/lib/transfer.c:869
#5 0x00007ffff78706f7 in Curl_readwrite (conn=0x615e20, data=0x603d80, done=0x7fffffffdeb2)
at /project/curl/lib/transfer.c:1071
#6 0x00007ffff787be6d in multi_runsingle (multi=0x60ccc0, now=..., data=0x603d80)
at /project/curl/lib/multi.c:1531
#7 0x00007ffff787c637 in curl_multi_perform (multi_handle=0x60ccc0, running_handles=0x7fffffffdfec)
at /project/curl/lib/multi.c:1808
#8 0x00007ffff787226e in easy_transfer (multi=0x60ccc0) at /project/curl/lib/easy.c:715
#9 0x00007ffff787240b in easy_perform (data=0x603d80, events=false)
at /project/curl/lib/easy.c:803
#10 0x00007ffff7872443 in curl_easy_perform (easy=0x603d80) at /project/curl/lib/easy.c:822
#11 0x0000000000401320 in main (argc=1, argv=0x7fffffffe258)
at /project/test/upload_file.cpp:65
(gdb) The valgrind output:
==21353== Memcheck, a memory error detector
==21353== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==21353== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==21353== Command: ./bin/upload_file-d
==21353==
==21353== Conditional jump or move depends on uninitialised value(s)
==21353== at 0x5160AE5: FormAdd (formdata.c:605)
==21353== by 0x5160FFB: curl_formadd (formdata.c:734)
==21353== by 0x4012F9: main (upload_file.cpp:61)
==21353==
==21353== Conditional jump or move depends on uninitialised value(s)
==21353== at 0x5185AB0: dprintf_formatf (mprintf.c:702)
==21353== by 0x5186AC8: curl_mvaprintf (mprintf.c:1070)
==21353== by 0x5165F4B: Curl_add_bufferf (http.c:1228)
==21353== by 0x51689D6: Curl_http (http.c:2465)
==21353== by 0x5182783: Curl_do (url.c:6186)
==21353== by 0x519772E: multi_runsingle (multi.c:1305)
==21353== by 0x5198636: curl_multi_perform (multi.c:1808)
==21353== by 0x518E26D: easy_transfer (easy.c:715)
==21353== by 0x518E40A: easy_perform (easy.c:803)
==21353== by 0x518E442: curl_easy_perform (easy.c:822)
==21353== by 0x40131F: main (upload_file.cpp:65)
==21353==
==21353== Conditional jump or move depends on uninitialised value(s)
==21353== at 0x5185B64: dprintf_formatf (mprintf.c:728)
==21353== by 0x5186AC8: curl_mvaprintf (mprintf.c:1070)
==21353== by 0x5165F4B: Curl_add_bufferf (http.c:1228)
==21353== by 0x51689D6: Curl_http (http.c:2465)
==21353== by 0x5182783: Curl_do (url.c:6186)
==21353== by 0x519772E: multi_runsingle (multi.c:1305)
==21353== by 0x5198636: curl_multi_perform (multi.c:1808)
==21353== by 0x518E26D: easy_transfer (easy.c:715)
==21353== by 0x518E40A: easy_perform (easy.c:803)
==21353== by 0x518E442: curl_easy_perform (easy.c:822)
==21353== by 0x40131F: main (upload_file.cpp:65)
==21353==
==21353== Use of uninitialised value of size 8
==21353== at 0x5185B32: dprintf_formatf (mprintf.c:729)
==21353== by 0x5186AC8: curl_mvaprintf (mprintf.c:1070)
==21353== by 0x5165F4B: Curl_add_bufferf (http.c:1228)
==21353== by 0x51689D6: Curl_http (http.c:2465)
==21353== by 0x5182783: Curl_do (url.c:6186)
==21353== by 0x519772E: multi_runsingle (multi.c:1305)
==21353== by 0x5198636: curl_multi_perform (multi.c:1808)
==21353== by 0x518E26D: easy_transfer (easy.c:715)
==21353== by 0x518E40A: easy_perform (easy.c:803)
==21353== by 0x518E442: curl_easy_perform (easy.c:822)
==21353== by 0x40131F: main (upload_file.cpp:65)
==21353==
==21353== Conditional jump or move depends on uninitialised value(s)
==21353== at 0x5185BFF: dprintf_formatf (mprintf.c:749)
==21353== by 0x5186AC8: curl_mvaprintf (mprintf.c:1070)
==21353== by 0x5165F4B: Curl_add_bufferf (http.c:1228)
==21353== by 0x51689D6: Curl_http (http.c:2465)
==21353== by 0x5182783: Curl_do (url.c:6186)
==21353== by 0x519772E: multi_runsingle (multi.c:1305)
==21353== by 0x5198636: curl_multi_perform (multi.c:1808)
==21353== by 0x518E26D: easy_transfer (easy.c:715)
==21353== by 0x518E40A: easy_perform (easy.c:803)
==21353== by 0x518E442: curl_easy_perform (easy.c:822)
==21353== by 0x40131F: main (upload_file.cpp:65)
==21353==
==21353== Conditional jump or move depends on uninitialised value(s)
==21353== at 0x5185C8E: dprintf_formatf (mprintf.c:756)
==21353== by 0x5186AC8: curl_mvaprintf (mprintf.c:1070)
==21353== by 0x5165F4B: Curl_add_bufferf (http.c:1228)
==21353== by 0x51689D6: Curl_http (http.c:2465)
==21353== by 0x5182783: Curl_do (url.c:6186)
==21353== by 0x519772E: multi_runsingle (multi.c:1305)
==21353== by 0x5198636: curl_multi_perform (multi.c:1808)
==21353== by 0x518E26D: easy_transfer (easy.c:715)
==21353== by 0x518E40A: easy_perform (easy.c:803)
==21353== by 0x518E442: curl_easy_perform (easy.c:822)
==21353== by 0x40131F: main (upload_file.cpp:65)
==21353==
==21353== Conditional jump or move depends on uninitialised value(s)
==21353== at 0x515EDB2: Curl_pgrsSetUploadSize (progress.c:247)
==21353== by 0x5168AF0: Curl_http (http.c:2500)
==21353== by 0x5182783: Curl_do (url.c:6186)
==21353== by 0x519772E: multi_runsingle (multi.c:1305)
==21353== by 0x5198636: curl_multi_perform (multi.c:1808)
==21353== by 0x518E26D: easy_transfer (easy.c:715)
==21353== by 0x518E40A: easy_perform (easy.c:803)
==21353== by 0x518E442: curl_easy_perform (easy.c:822)
==21353== by 0x40131F: main (upload_file.cpp:65)
==21353==
==21353== Conditional jump or move depends on uninitialised value(s)
==21353== at 0x5164FE1: http_perhapsrewind (http.c:452)
==21353== by 0x5165234: Curl_http_auth_act (http.c:547)
==21353== by 0x5169F6C: Curl_http_readwrite_headers (http.c:3140)
==21353== by 0x518B51A: readwrite_data (transfer.c:480)
==21353== by 0x518C6A7: Curl_readwrite (transfer.c:1062)
==21353== by 0x5197E6C: multi_runsingle (multi.c:1531)
==21353== by 0x5198636: curl_multi_perform (multi.c:1808)
==21353== by 0x518E26D: easy_transfer (easy.c:715)
==21353== by 0x518E40A: easy_perform (easy.c:803)
==21353== by 0x518E442: curl_easy_perform (easy.c:822)
==21353== by 0x40131F: main (upload_file.cpp:65)
==21353==
==21353== Conditional jump or move depends on uninitialised value(s)
==21353== at 0x5164FEB: http_perhapsrewind (http.c:452)
==21353== by 0x5165234: Curl_http_auth_act (http.c:547)
==21353== by 0x5169F6C: Curl_http_readwrite_headers (http.c:3140)
==21353== by 0x518B51A: readwrite_data (transfer.c:480)
==21353== by 0x518C6A7: Curl_readwrite (transfer.c:1062)
==21353== by 0x5197E6C: multi_runsingle (multi.c:1531)
==21353== by 0x5198636: curl_multi_perform (multi.c:1808)
==21353== by 0x518E26D: easy_transfer (easy.c:715)
==21353== by 0x518E40A: easy_perform (easy.c:803)
==21353== by 0x518E442: curl_easy_perform (easy.c:822)
==21353== by 0x40131F: main (upload_file.cpp:65)
==21353==
==21353== Invalid read of size 8
==21353== at 0x5162138: Curl_FormReader (formdata.c:1453)
==21353== by 0x516204A: readfromfile (formdata.c:1413)
==21353== by 0x516217E: Curl_FormReader (formdata.c:1458)
==21353== by 0x518AD6C: Curl_fillreadbuffer (transfer.c:118)
==21353== by 0x518C167: readwrite_upload (transfer.c:869)
==21353== by 0x518C6F6: Curl_readwrite (transfer.c:1071)
==21353== by 0x5197E6C: multi_runsingle (multi.c:1531)
==21353== by 0x5198636: curl_multi_perform (multi.c:1808)
==21353== by 0x518E26D: easy_transfer (easy.c:715)
==21353== by 0x518E40A: easy_perform (easy.c:803)
==21353== by 0x518E442: curl_easy_perform (easy.c:822)
==21353== by 0x40131F: main (upload_file.cpp:65)
==21353== Address 0x121 is not stack'd, malloc'd or (recently) free'd
==21353==
==21353==
==21353== Process terminating with default action of signal 11 (SIGSEGV)
==21353== Access not within mapped region at address 0x121
==21353== at 0x5162138: Curl_FormReader (formdata.c:1453)
==21353== by 0x516204A: readfromfile (formdata.c:1413)
==21353== by 0x516217E: Curl_FormReader (formdata.c:1458)
==21353== by 0x518AD6C: Curl_fillreadbuffer (transfer.c:118)
==21353== by 0x518C167: readwrite_upload (transfer.c:869)
==21353== by 0x518C6F6: Curl_readwrite (transfer.c:1071)
==21353== by 0x5197E6C: multi_runsingle (multi.c:1531)
==21353== by 0x5198636: curl_multi_perform (multi.c:1808)
==21353== by 0x518E26D: easy_transfer (easy.c:715)
==21353== by 0x518E40A: easy_perform (easy.c:803)
==21353== by 0x518E442: curl_easy_perform (easy.c:822)
==21353== by 0x40131F: main (upload_file.cpp:65)
==21353== If you believe this happened as a result of a stack
==21353== overflow in your program's main thread (unlikely but
==21353== possible), you can try to increase the size of the
==21353== main thread stack using the --main-stacksize= flag.
==21353== The main thread stack size used in this run was 8388608.
==21353==
==21353== HEAP SUMMARY:
==21353== in use at exit: 83,286 bytes in 131 blocks
==21353== total heap usage: 320 allocs, 189 frees, 101,211 bytes allocated
==21353==
==21353== LEAK SUMMARY:
==21353== definitely lost: 0 bytes in 0 blocks
==21353== indirectly lost: 0 bytes in 0 blocks
==21353== possibly lost: 365 bytes in 8 blocks
==21353== still reachable: 82,921 bytes in 123 blocks
==21353== suppressed: 0 bytes in 0 blocks
==21353== Rerun with --leak-check=full to see details of leaked memory
==21353==
==21353== For counts of detected and suppressed errors, rerun with: -v
==21353== Use --track-origins=yes to see where uninitialised values come from
==21353== ERROR SUMMARY: 26 errors from 10 contexts (suppressed: 10 from 6)
The good news is I narrowed it down.
The "ANYSAFE" auth method can cause a form to be reposted from scratch.
In this situation, the user-provided readfunction callback is switched to the standard fread callback, and then called with the user-provided readdata pointer, which the fread callback is incorrectly casting and writing to.
Valgrind also has a heap of warnings.
First, is the gdb output showing when the fread_func is reset, and then called.
Note: readstream is my user callback function.
Curl_FormReader is the built-in callback function
Curl_FormReader casts my user callback data 0x121 and then tries to access it.
Then, the code.
It has lots of odd variables and padding, I used them to force a segfault.
I believe your compiler may not crash in the same way, so I'm attaching a lot of output.
Then, GDB and Valgrind output
The code:
The GDB output:
The valgrind output: