Support RAW files (DNG, NEF, CR2, etc..)

[trednx]

Hello
Is it possible to add DNG support? Each DNG has embedded preview, so I think, it will be not very resource "hungry" for RPI to show this preview without real-time rendering :)
Thanks for your wonderful app!

[bpatrik]

Hi,

I was looking into supporting raw photos a few month earlier and there was not any simple solution for that.

I'd like to add the support of raw photos eventually, but not sure when.

Patrik

[ch33zer]

I'd like this support for CR2 files as well.

Until this is supported my plan was to use exiftool and cron to periodically extract the embedded thumbnail and store it alongside the raw file. This obviously requires more storage and it would be better if pigallery supported it out of the box.

I have a few thoughts for implementing this:
1)I know that pigallery has a tmp directory, would it be possible to, during indexing, extract the thumbnail and store it in tmp then serve that in place of the actual file?
2) Alternatively, upon receiving a request for a raw file rather than serving the file itself could we extract the thumbnail and just return that without even storing it anywhere?

I'm sorry if these are ignorant suggestions, I haven't actually looked at the code in depth to see what's possible. If I find some time I may work on this.

[bpatrik]

My plans to have raw photos support would be:

1. remove the GM and Jimp based thumbnail generation (that would increase the complexity by a lot). Keep sharp as that is the fastest
2. Add extensions you would like to support with converting/extraction here:
   

   pigallery2/src/common/SupportedFormats.ts

   Line 24 in b6f93ac

   Photos: <string[]>[],

3. check which of them cannot be handled by sharp (check multiple versions of CR2, DNG files as some of them work out of the box, some of them not)
4. find a node module that supports CR2, DNG converting (preview extracting) (should be fast, run on multiple platforms and have the least external dependencies)
5. Implement the conversion / (preview extracting here):
   https://github.com/bpatrik/pigallery2/blob/master/src/backend/model/threading/PhotoWorker.ts
   PhotoWorker works with these inputs:
   

   pigallery2/src/backend/model/threading/PhotoWorker.ts

   Lines 46 to 59 in b6f93ac

   	export interface RendererInput {
   	type: ThumbnailSourceType;
   	mediaPath: string;
   	size: number;
   	makeSquare: boolean;
   	outPath: string;
   	qualityPriority: boolean;
   	cut?: {
   	left: number,
   	top: number,
   	width: number,
   	height: number
   	};
   	}

   
   (This is the expected output image)

I wont have time for this in the near future, but feel free to jump in :)

[lyonzy]

Hey guys,

I had a bit of a crack at this and managed to get it working for ORFs (Olympus) - albeit with some hacks.

I used raw-preview which is a bit undermaintained but very simple (native interface to exiv2). I also tried exiv2node but I couldn't get it to build.

What was required was fairly simple:

• SupportedFormats change as above
• This in PhotoWorker:

  private static async getRawPreview(filename:string):Promise<Buffer> {
    // n.b. make sure you require sharp before raw-preview otherwise you'll get this:
    // https://github.com/lovell/sharp/issues/1428#issuecomment-432558430
    var rp = await RawPreview.getPreviewAndMetadata(filename);
    return new Buffer(rp.image.base64, 'base64');
  }

• And this inside the worker for Sharp (didn't do the other ones at this stage):

      if (input.mediaPath.toLowerCase().endsWith('.orf')) {
        Logger.silly('[SharpRenderer] rendering photo from raw preview:' + input.mediaPath + ', size:' + input.size);
        image = sharp(await this.getRawPreview(input.mediaPath), {failOnError: false});
      } else {
        Logger.silly('[SharpRenderer] rendering photo:' + input.mediaPath + ', size:' + input.size);
        image = sharp(input.mediaPath, {failOnError: false});
      }

• MetadataLoader needed to be updated to get metadata for the ORFs, otherwise they'll all appear squished square, and not rotated based on Orientation. Luckily for me ORFs are just TIFFs with a different header so I could just change the header and process them as TIFFs with ExifReader:

              if (fullPath.toLowerCase().endsWith('.orf')) {
                  data[2] = 0x2a;
                  data[3] = 0x00;
                  var exifTags = ExifReader.load(data);

                  exif = {
                      tags: {
                          // this is a different value in ExifReader
                          ISO: exifTags.ISOSpeedRatings.value
                      }
                  };

                  // pull all these out of the .value field, and remove a few unnecessary big ones
                  for (var tag in exifTags) {
                    if (tag != "PrintIM" && tag != "MakerNote" && tag != "UserComment") {
                        exif.tags[tag] = exifTags[tag].value;
                    }
                  }
              } else {
                  exif = ExifParserFactory.create(data).parse();
              }

• I also needed to add this because the EXIF data I had didn't have any of the dimensions MetadataWorker was looking for:

              else if (exif.tags.ImageWidth && exif.tags.ImageLength) {
                metadata.size = {width: exif.tags.ImageWidth, height: exif.tags.ImageLength};
              }

The ORF/TIFF approach probably isn't suitable long-term so you might choose to use raw-loader (it gets metadata too) or even better you might be able to pull it out of the exif using ts-exif-parser or ExifReader. However this relies on:

• knowing where find exif in all the raw formats you want to support
• knowing how to get the previews out of that exif (I couldn't find anything in the ORF files that was obviously a JPEG - I assume it's in MakerNote somewhere)

Obviously you'd need to find a way of knowing which files to apply this to that isn't path.endsWith('.orf') :) I guess either a list of supported formats for the raw-preview library/exiv2 or another field in SupportedFormats should do the trick.

Also an interesting note is that I also managed to get RAWs working with gm - I had to install imagemagick in the container I was using and then tell gm to use it (subClass({imageMagick: true}); ). This doesn't solve the metadata problem though, and it was fairly slow because I believe it was rendering them rather than extracting the preview (although I should note I ran it on a fairly slow box and my ORFs were on a NAS).

Hope this helps :)

[bpatrik]

Lets make this bug about supporting raw files. (renaming ticket to reflect is)

[bpatrik]

In the meantime point 1. in #127 (comment) is done.

The app uses sharp. From a quick google search I found, that sharp does not supports raw files:
lovell/sharp#616
But it sues libvips in the background that can be somehow tweaked: libvips/libvips#537
It would require a more in-depth look to figure out.

[jezikk82]

Any chance we can see this feature in near future ?

[bpatrik]

no, I do not think so, unless someone implements it

[bpatrik]

I did a search and it seems (without looking into details too much) that sharp (the used image manipulation lib) can support raw files if we build vips ourself.

If i understand correctly changing vips to use magicload will allow us to open some raw files.

Some links:

1. Is it possible to use magickload? lovell/sharp#2852
2. Does Vips supports RAW files like NEF,CR2 files libvips/libvips#255

If my understanding is correct only the following changes are needed:

1. Build vips in the docker container with magicload.
2. Add the raw photo extensions in the settings.

So no code change is needed.
This will already give a good enough (but still beta) experience

For a full solution a significant thumbnail and photo refactoring isneeded, without it zooming wont work with raw photos.

Would be appreciated if someone could figure out the vips building part in the Docker file

Well, first of all, I am no developer at all. Just reading through this thread and a couple of more it seems like this issue here is relevant, another app but it uses sharp as well.

If I understood it right, the solution is to build sharp from Alpine with libraw-dev vips-dev vips-magick. vips-magick instead of vips to get magickload.

This is the repo from @uhthomas explaning how he did it.

I can't do it myself beacuse I don't know how but I am happy to test it in case this is the way to go.

Sorry, I was meant to reply to @bpatrik with this comment.