terminate called after throwing an instance of 'std::bad_alloc' on video file smaller than 4GiB

[sebma]

Hi,

I have build untrunc and run it on Ubuntu 14.10 and here is what I have :

$ untrunc REFERENCE.MOV BAD.MOV
Reading: REFERENCE.MOV
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted (core dumped)

When running it without arguments it prints the following usage :
Usage: untrunc [options] <ok.mp4> [<corrupt.mp4>]

Can you tell the possible options for your program to be more verbose ?

Running movdump.exe -i BAD.MOV -o BAD-REPAIRED.MOV -nfd -ref REFERENCE.MOV >> movdump.txt on Windows repaired the video but I don't want to be stuck to Windows only for that.

UPDATE : My video file is much smaller than 4GiB

Can you help ?

[maksadbek]

I have the same issue

[jensp]

untrunc seems to be loading the entire videos into RAM. For me that was ~400GB of video data. I worked around the issue by creating a large swap file on a fast SSD, but it would be neat if untrunc would load large files incrementally.

[sebma]

I don't understand because at the time of my test my swap had (and still has) about 12GiB free which should be enough :

$ swapon -s | column -t
Filename   Type       Size            Used     Priority
/dev/dm-7  partition  12582908  127644  -1

movdump.exe on Windows didn't need such a big Pagefile to do the job.

Can you please look into this issue ?

[DaveQB]

I have this issue too.
http://paste.ubuntu.com/14884336

I did sound like a memory issue, but I have 10GB of free memory. The good file is 17GB and the truncated file is 12GB.

I'll try the swap creating solution. I guess it only loads 1 video as a time into memory?

[cdemoulins]

I had this issue and it can be solved using 'unsigned int' for all 'start', 'length', 'offset' variables.

I will try to send a merge request.

[valioDOTch]

@cdemoulins As far as I can tell you submitted a merge request that was merged into the master.
Is that correct?

Unfortunately I still got that error with any mp4 file, despite having loads of swap memory, that was exceeding both video files.

The "workaround" that fixed it for me was using a much shorter video clip for the "working video".

[rysiekpl]

I ran into the same issue. For some reason untrunc tries to allocate ~16EiB (~16 × 1024^16 bytes) of memory:

mmap(NULL, 18446744073349910528, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)
mmap(NULL, 18446744073350045696, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = -1 ENOMEM (Cannot allocate memory)

...and inevitably fails with the error in question.

[sebma]

@rysiekpl Hi, geat ! It seems you have found why the pb. crashes :) !
Can you please submit a merge request or ask ponchio to do so ?

[oernii]

Just letting you know, I changed all "start" and "offset" from int to unsigned int and recompiled. Did not help, it still tries to mmap 16exabytes.

[lletourn]

It's more complicated than that. If your video is bigger than 4Gb, you also need to change the offset vectors from vector to vector<int64_t>, the chunk method too and everywhere where there is an
int offset...

I've started to do this with some success, but I think there is also an issue in atom.cpp.
it used readInt to read co64 offsets which are 8bytes not 4...

basically I think, but might be wrong, there is a lot of work to have this work with videos >4G.

It would also be nice to buffer the mdat reading of the correct video instead of trying to load it all in ram...

[sebma]

My file is not even 1GiB. Why not create a separate issue for files bigger than 4GiB ?

[lletourn]

Oh you didn't mention the size of your video. Everybody else was giving out files bigger than 4GB.

I will open another ticket.