Sensor calibration correction parameter -s

Mid of december 2013 I received an email by a user of m8raw2dng showing me his images that had been shot pretty much in the dark, at nighttime in snow covered norway. Unfortunately the images were split into two different sensor halves of different brightness. He suggested that the sensor calibration might have been skipped when using RAW files.

Darkfield correction -sd

I checked a black image of my M8 (taken against lens cap)darkfield_uncorr and to the right you can see what I got. not only are the two halves of different brightness, but also there are variations from left to right in the halves themselves. (horizontal banding is a different topic though, which I will not go into detail here…)

Luckily the pattern did not vary much from image to image, so I had the idea to use such an image as input for estimating of correction values.

So you need a dark frame image of your M8. I always recommend to use a medium exposure time, so not 1/8000s and not 4s. I take 1/60s most of the time. I have not tested if that has a big influence though… Take that image as RAW+JPG Fine against your lens cap, to be sure it is dark. ISO 160 of course.

Sensor database creation

If you have bright vertical pixel lines in your image I recommend to fix them first (see further below).
Do the conversion of your dark image with the additional parameter -sd (for sensor darkfield creation). The tool will analyse the image and save correction to the file lensdb.ini creating a new entry and/or file if either doesn’t exist. The reference of the entry is the serial number of your camera, so if you have several M8 cameras, you could convert them all with the same lensdb.ini. But you have to do the darkfield generation at least once for each one.

Sensor darkfield fixdarkfield_corr

This will be done automatically if you run the tool with the -s parameter (NOT -sd as it will overwrite your previous calibration). The tool checks the lensdb.ini if there are correction values stored. If so, the image will be corrected. The image on the left shows the same one as above but with the correction applied.

Vertical pixel lines

I had several users complain about bright vertical lines showing up on their converted DNGs that were not visible in the standard M8 DNGs. Those lines can either be cyan or orange coloured, depending on the column on the sensor they affect. In most cases there is a stuck pixel at the top of the problematic part of the column. This will hinder correct readout of that column and all pixels below the pixel will be much darker than the surrounding pixels. So if the affected line contains green and blue pixels, the interpolation makes that line bright red, as there are no green and blue value to match the brightness and for the columns with red and green pixels vice versa.

Step by step fix process

So first you need a sensdb.ini file, as the correction information is stored there. Either m8raw2dng has created one for you as you went through the darkfield correction described above, or you have to create a new one.

The file for one faulty sensor line looks like this (stuff after % is explanation, don’t put that in the file):

[3010101]                             % the identifier = your camera’s serial number
Line = 935                           % the x-coordinate of the upper line point
Line = 1700                         % the y-coordinate of the upper line point
Line = 935                          % the x-coordinate of the lower line point ( same as upper)
Line = 2645                         % the y-coordinate of the lower line point (usually to end = 2645)
LevelCorrection = 4.242321       %If you did the sensor darkfield correction, you’ll have a lot of those values from here on…
LevelCorrection = 4.132235
LevelCorrection = 4.113313
...

So as you can see, one line has four entries, because a line is defined by two points that themselves have an x and y coordinate. In images the x coordinate goes from left to right, the y coordinate from top to down. So the point (x=0, y=0) is the upper left corner, the point (x=3967,y=2645 is the lower right corner. The two x coordinates must be the same, otherwise it’s not a vertical line.

All you have to do is to get the coordinates of the faulty line on your sensor. I have successfully used two different ways to do so…

The straightforward method:

  1. Export the DNG with the faulty line clearly visible as TIFF, JPG, BMP, etc.
  2. Open in it in an image editing program like Photoshop or GIMP (even MS Paint is fine).
  3. Most programs have a feature where they show the mouse coordinates as you hover over the image. Go to start and end point of the bad line and note down the coordinates.
  4. The actual RAW file contains 2 pixels more around the image borders (DNG 3964×2642, RAW 3968×2646). This is for interpolation purposes. You have to add those 2 pixels shift to all the coordinates (x and y) you wrote down.
  5. Open (if it exists) or create lensdb.ini in a text editor.
  6. Follow the layout description above (first line camera serial no) and add four lines with Line = x1, Line = y1, Line = x2 and Line = y2, where x1,y1 belong to the upper line coordinate and x2,y2 belong to the lower line coordinate
  7. Save the file at the same location as the m8raw2dng executable
  8. Run the conversion with parameter -st (sensor test mode)
  9. The resulting DNG should now have a really bright line where your faulty line was. If so you did everything right. :)
  10. Run the conversion with parameter -s, the line should be gone in the resulting DNG.
  11. If you have done the darkfield correction before patching the line I recommend to redo the darkfield correction.

The iterative method:linefix_steps

  1. Look at the position of the line in your image and roughly estimate the coordinates. (first part of the image to the right)
  2. Rerun the conversion with -st (testmode) to get a bright line at your estimated position (second part of the image to the right)
  3. Check the image again and refine your estimation. I recommend to first get the y coordinate right, then move the x
  4. Repeat steps 2 and 3 until the test line covers the faulty line (takes usual 2-3 tries) (third part of the image to the right)
  5. Run the conversion with parameter -s, the line should be gone in the resulting DNG. :) (fourth part of the image to the right)
  6. If you have done the darkfield correction before patching the line I recommend to redo the darkfield correction.

 

If you encounter more than one bad pixel line on your images, you can just add another set of four Line = ... rows to the sensdb.ini doing exactly the same steps as for the first correction line to get the coordinates.

You do this once and it will be fixed in all your converted images from then on. So it’s worth the effort. And should you ever get a new line, instead of having to send it to Leica for a sensor remap, you can fix it yourself in a few minutes.