Scanner calibration

Okay, maybe I should clarify the project I'm attempting and where I am.

20 years ago I was a colorblind stamp-collecting computer tech who thought 'hey, wouldn't it be cool if I could use a scanner to determine color varieties!' Naïve me. Without even accounting for the differences between human color perception and machine color perception, the consumer-level tech at that time just wasn't workable. And my stamp collection got put away around then because that's about the time my ex started to decide she wanted to be my ex.

Now I'm a graduate student working in the field of psychophysics, with a renewed interest in those stamps I put away. Although I specialize in depth perception, my background in color perception and spectral analysis is pretty in-depth. Plus, as long as I have access to the vision lab I'm working in, I can test consumer-level equipment with research-level equipment. The current level of consumer tech looks to be very close to being up to the original task. It seems to me that, right now, with a couple thousand dollars and a programmer, I can build a desktop system that will do a pretty decent job of accurately encoding and displaying most (if not all) listed stamp color varieties. In a couple more years (once 4k gets small enough to put on laptops), I bet I can do it portably (bring it to stamp shows, dealers, etc.).

But first I want to profile the equipment I have now and see how variable it is over time. Hence the query with regard to easy-to-use calibration software. What I have is kinda painful to use once, much less monthly, weekly, or daily.

And, right now I do everything on a laptop - which has no color profiling for the monitor to speak of - so if there's a calibration system out there, I'd be grateful if someone would let me know. In the lab we map our monitors with a photometer, calculate the RGB gun algorithms, and drop them into Matlab. Although it's not really very practical for my laptop, it at least shows it can be done with a software.

Thanks,

Andy

BTW, I have a feeling I'm going to be asking a LOT of questions like these over the next few months - I'm not as up-to-date on some of this stuff as I'd like to be, and those of you with a lot of time in doing color studies or what-not, if you see me say something stupid or impractical, please correct me. 'What I think should happen' will only become 'what does happen' with a lot of nudging!  ;-)  I'll keep posting anything new or interesting within this thread so people who want to ignore it can do so readily (unless and until admin recommends something different).

Jun 1, 2018 11:52:48 GMT brightonpete said:

Jun 1, 2018 6:27:43 GMT ajkitt said:

And, right now I do everything on a laptop - which has no color profiling for the monitor to speak of - so if there's a calibration system out there, I'd be grateful if someone would let me know. In the lab we map our monitors with a photometer, calculate the RGB gun algorithms, and drop them into Matlab. Although it's not really very practical for my laptop, it at least shows it can be done with a software.

BTW, I have a feeling I'm going to be asking a LOT of questions like these over the next few months...

Talk about a complicated setup. The only thing I know of is something like FStoppers Review of a monitor calibrator. But I'm sure you already know of these things.

Ask away, we can always TRY. Not sure the answers for you will be forthcoming though!

That's just it, though. My consumer-tech knowledge for the last 20 years has been pretty weak, and the info all y'all have provided so far has been a tremendous help at getting me up to speed. Everybody who has followed up on this thread so far has posted great information. You weren't the only person to mention the Spyder, but the weblink you provided had a walk-through that was hugely informative! It told me that the Spyder does exactly the kind of profile embedding I thought was needed, and the people who built it have a very good idea of the problems inherent in trying to match a 3-color luminous display to what people 'see' in the real broad spectrum analog world. It seems to be a well-thought device, so you all have helped me put it really high on my discretionary budget!

And once I get the thing, I can test its results using lab equipment. By seeing the walk-through, I can make an educated guess with regard to its limitations. Once I have one, though, I can see how much those limitations will affect the desired 'stamp-collector outcome.'

A lot of people might think I'm just being nit-picky, but the current mainstream (24 - or 32, for that matter -bit sRGB) color system CANNOT display half the colors humans see. And it gets worse when you realize that there are a lot of people who have color vision problems and don't know it. 10% of males can't pass a simple go-nogo colorblind test (like those Ishihara dot tests). But when it comes to actual sensitivity tests (like the Munsell 100 hue), the numbers get MUCH worse. The bottom line is that if you're using standard sRGB and think the colors are good, you are probably at least a little color-insensitive.

Just sayin'.

A couple people wanted to see images, so here they are. Of each pair, the first is a calibrated example and the second is scanned using the scanner's default settings. BTW, the defaults were all scanned at the same time - I put them all on at the same time and cut them out, so you can see the variability between them. The calibrated versions, I don't know if they came off the same scan or not. Those are the images I use for color analysis, and I usually scan well upwards of 50 at a time.

And to make sure the calibrated scans are pretty good, I had a "super-discriminator" at the lab help me with monitor calibration (as far as we could go using the native utility), and she confirmed that yes, the final result was a very good color representation within the limits of the lighting in the lab.

The differences in apparent brightness were likely due to turning off the automatic pedestal and gain features of the scanning software, but the actual hue changes were almost certainly a function of calibration.

Jun 2, 2018 0:58:13 GMT Ryan said:

These articles do nothing to answer your questions about scanner calibration but I think they're interesting in their own right.  They give a good indication of how a scientific approach can be used to determine shades.  I await the day when the price point of a spectrograph drops a bit (as in, "divide the current price by about 200 ....").

The Colors of Martin Van Buren: An Engraved Postage Stamp (1938-1959) - Diane DeBlois & Robert Dalton Harris
Proposal for a Colorimetric Analysis Methodology for Philatelic Studies - John H. Barwis
Philatelic Shade Discrimination Based on Measured Color - David L. Herendeen, James A. Allen & Thomas Lera

The last article was originally published by the Institute for Analytical Philately.  Their website has a brief bibliography for other articles on colourimetry.  Publications of their first two symposia are also available in PDF format.  Lots of this stuff is way outside of my interest range (or way over my head) but they're nice additions to my philatelic library.

Proceedings of the First International Symposium on Analytical Methods in Philately
Proceedings of the Second International Symposium on Analytical Methods in Philately

I look forward to learning about whatever solutions you find for your scanner questions - I'm currently using an older all-in-one scanner that does the job but does a poor job of reproducing some colours, and at some point in the near future I'll finally update my scanner.  For whatever reason, it comes pretty close with both blue & red but misses badly with purple - how is that even possible?  After scanning, I use a colour levelling preset which I created in GIMP to get the white & black balance as close to the stamp's true colours as is possible.  It doesn't fix the colour balance problems with purple & yellow in particular, but oh well, it makes the paper look like paper instead of white enamel, which is what the original scan often looks like.

Ryan

I ran into the Herendeen article online, and it's pretty much why I wanted to take a shot at this in the first place! The problem with the article for me was that it went straight from colorimetry to expert perception without talking about graphically displaying anything. Reading between the lines, it really emphasizes the problems of color separation and identification, though - no two stamps are the same, and there can be a lot of overlap and "interpretation" going on.

I wasn't aware of the Barwis thing, but that one is a little more end-oriented - create a standardized color system that anyone can use. But it still ignores the display problem. It says it's a proposal, does anyone know if it was actually done? I'm going to have to fish around the Analytical Philately website to see, I suppose. Very cool website and info!  Thanks! I had never heard of the group before. I also think it's cool that these papers are all dated within the last 10 years, which suggests to me that that's about when the tech caught up to the problem.

So, simplified version of the issue:

Right now, consumers have access to some pretty decent colorimetry equipment; that's essentially what a scanner is - it converts broad spectrum color data into 3-point (RGB) metamers (looks the same to the human eye as the source color, but makes use of different colors to achieve that). In order to do that well, however, the scanner has GOT to be calibrated. Without calibration, a scanner outputs the wrong numbers and anything that comes from those numbers will be flawed. And there seems to be handheld CCD scanners out there, so this tech appears to be mobile (assuming decent resolution).

Right now, consumers have access to some pretty decent display equipment too. The 4k system has an awesome color range, but it's just not available on laptops yet as far as I could tell. When it becomes available on laptops, this tech is mobile.

The 4k system exists, and the color space it's capable of is pretty much defined in rec. 2020 (there may be others, but I know that one exists, and it's definitely adequate). Somebody's got to have software that converts images into the 2020 colorspace. If not, 4k would be kind of irrelevant, right? And, if it's not available in scanner equipment yet, it certainly will be soon. After that, it's just a matter of scanning a whole lot of known examples to come up with numerical approximations of color varieties, and with a properly calibrated 4k monitor you'll have a good color representation with which to physically compare stamps you come across at shows, dealers, what have you. Online sales, however, will always be risky - there would always be the problem that you don't know if the seller is using the right equipment, calibrated properly, in generating his images.

So the last problem is making all the hardware and software easily useable. That was pretty much what my first post in this thread was about. I did manage to find some decent calibration targets for less than $50 (Wolf Faust at www.targets.coloraid.de/ sells them for between $6 and $10 a pop, depending on quantity), but finding free or cheap (and easy to use) calibration software isn't working so well for me. There is some great freeware, but it is command-line based and not easy to use. The stuff with a user interface I've seen is $70 and up, and I don't want to pay $70 just to find out whether or not some software is any good!

One way or another, SOMEbody is doing this stuff. Myself, I'm enough of a control freak to want to do it myself!  ;-)

Jun 4, 2018 6:41:47 GMT Ryan said:

Jun 3, 2018 15:01:49 GMT ajkitt said:

I'm going to have to fish around the Analytical Philately website to see, I suppose. Very cool website and info!

Their section on IAP Presentations includes a PowerPoint document in which mention is made, italicized and in red, that "scanners can be used to identify colors IF preliminary analysis has been performed".  A couple of slides later in the presentation, there is reference to another article by Herendeen, Allen & Lera - "Using a Scanner to Determine Stamp Shades", published apparently in both The London Philatelist and The Collectors Club Philatelist.  I haven't been able to find it online, though.

An article written by Timothy A. Lyerla entitled "Scanning for Stamp Shades" was published in the July 2014 edition of American Philatelist.  This forum is a chapter member of the APS so its members are able to read copies of American Philatelist but I think you'll have to graduate through the 2 week / 50 post process to be able to see it.  Once you do, have a look at this post for access to it.  In the meantime, you can have a look at the text portion of the article here - none of the images from the article are included.  On a quick look through it I didn't see any attempt at all to deal with scanner calibration, so again it might not pique your interest, but nonetheless it covers some of the methodology of analyzing an image file to identify colour shades.

Ryan

It ALL piques my interest - the calibration info is part of a goal, not the mission itself. But scanning is the first step so it's hard to talk about much else before that's resolved. Maybe I'll post some info about the calibration process using Argyll so all y'all can see what a pain it really is to use - there's no fast testing of the different switches (a problem with using the program), and it's hard to confirm the output does what it says it does (more of a global scanner problem - that "preliminary analysis" from the IAP thing, I assume). Once the fall semester starts, I think I'm going to talk to a few computer science teachers to see if they have any grad students looking for a topic for their Masters thesis.

Until then, I wonder if I can talk Admin (is that tomiseksj?) into granting me an early graduation if I promise to write some of this up for the newsletter so I can get at those articles? ;-) Or maybe I'll actually pay the APS dues, right? I remember seeing some kind of discount coupon on this forum somewhere... . I'll also see if I can pull a digital inter-library loan from work for the articles. I am constantly surprised at the obscure stuff that's available through the library system. And it's not just the University system, either. My local library is part of the same ILS system, and also has free access to EBSCO. Some stuff is definitely University-level only, though (JSTOR, for example, is a huge repository of academic articles that I can get at for free through school).

I've also been fishing around photography websites, and it looks like they've been using something called ProPhoto RGB with 16 bits per channel, which they say covers all the colors in the natural world (which might just be a reflected versus luminous source thing). The gamut profile images out there look very good. I didn't find any actual references, but it looks like they've been using this for a while (before Adobe RGB, even). Just goes to show that there are no new ideas, just new applications. And, apparently, even the application isn't new. Just new to me!  :-)

Thanks for the email, Steve.

I just read the Lyerla article and although it's very cool in its own right, it's not quite what I'm after. The good part is that what he did can be used to compare shades of the same design whether you calibrate or not. The downside is that his technique can't be used to establish a metric for color descriptions. Hopefully I'll be able to read a few more of the items Ryan pointed out to me this weekend.

And, for all those who noted that 4k *is* available on laptops, I just did not know that. That would be me yet again a couple years behind the tech curve! So, standing corrected here, everything a person needs to do mobile color assessments is - in theory anyhow - available right now!

I thought about getting VueScan when I last did a driver hunt for my 1660 - I mean, it looks like it does all the right things (TIFF & RAW settings, IT8, etc.) but they kind of all do that. And, when I found a driver that worked, it was free, so why spend the money? But now, after all my hunting and digging on scanning, I find the 1660 was at the edge of obsolescence for what I want it to do, so picking up something with better optical resolution seemed like a good idea. I'm sure the 1660 failed on me because some MS auto-update interfered with my driver fix and that VueScan would have resolved it, but for the same price I'm getting some seriously overkill resolving power - something that no software cure can do. But I do keep VueScan in the back of my head because sooner or later the v30 won't have drivers available either. I mean, 4800 dpi optical? It staggers me. That's not resolving power anymore. That's magnification!

And (doug360), the hologram thing is interesting. A CCD scanner does collect some limited depth information (from the light source offset), so you could get some value from that (embossed stamps) but doing the math to derive depth metrics from shadow effects seems to be more pain than gain to me. Then again, all you have to do is come up with the depth algorithm once, right? The storage benefits, however... I don't know. I think with holograms information of the whole is stored complete, but the resolution of the complete degrades with the relative size of the stored sample. If so, how's the storage gain an improvement over what you get from scaling regular image formats? Like I said, I don't know. I just don't know holography well enough, and I'm spending too much of my free time looking at color capture.

As for back to school, I'm a grad student at 55, and I've had several over-60s in classes I've taught. If you have the time and money, I HIGHLY recommend taking classes in topics that interest you. Older students do much better than their younger counterparts. And, between work ethic and real-life experience, they are an absolute joy to the professors! If you're game, talk to someone at a nearby university to see which classes you might need as preliminaries, and dig in! We aren't getting any younger, and - my perspective, anyhow - at the end of my life I'd rather regret the things I did than the things I wanted to do but didn't!  :-)