Still more DJVU

I’ve scanned and created another document, an article from the Gleanings for the ATM column that ran for years in the magazine Sky and Telescope. This article is by Dick Buchroeder and discusses the plans for a Catadioptric Herschelian Telescope, an interesting telescope design with tilted optics.

If you aren’t interested in telescope making, you might be interested in looking at the quality of the output images anyway. For the majority of the text (all but the first cover page), I just used the Any2Djvu server, but I goofed around a bit with the cover image (the color scan of the cover for the Sky & Telescope). I took the color scan, converted it to a grayscale map, and then thresholded it to produce a binary image that contained the text and some cruft from the cover image of the two watches. If you try to use the cjb2 compressor on this image, you don’t get very good results, but if you paint out the crud, and then use the djvumake program, you can get a very nice image, and one that is entirely comparable in size to the output of the dedicated server.

I’ll probably write up a simple guide on how to do this later, and see if I can
get it posted someplace where more people will see it and hopefully use it.

More experiments with djvu

I did a bit more research on djvu, and have done some more experiments. Long ago, I rescued a huge number of old Scientific American magazines from the dumpster of our local library. I grew up reading the Amateur Scientist column conducted by C. L. Stong, and of course Martin Gardner’s superb Mathematical Games. I ended up redonating most of those magazines to another library (who probably prompty threw them out).

Nevertheless, I do still have a few that include topics of interest, and decided to use one of them as source material for my experiments. In November 1981, the topic was pinhole cameras. I disassembled my copy of Scientific American and fed it to my $50 scanner, generating six 25 megabyte scans. I then converted these to reasonable quality JPEG images, and uploaded them to the
Any2DJVU service that is available on djvuzone. I took the individual pages and downloaded them, and using the djvm program, assembled them into a single downloadable document, which you can see here.

The resulting compression is 550:1. Not too shabby.

This compression seems to be hard to achieve using purely free tools on my machine. In particular, the free tools don’t include a tool for performing the segmentation that makes the overall process tractable. I’ve been toying around with crudely doing the segmentation by hand, but the results aren’t as good as the automatic version.

Anyway, I may convert some other of my available documents into this format
for redistribution, including some hard to find and useful telescope information.
Stay tuned.

From the Hackers mailing list…

“Of course the people don’t want war. But after all, it’s the leaders
of the country who determine the policy, and it’s always a simple
matter to drag the people along whether it’s a democracy, a fascist
dictatorship, or a parliament, or a communist dictatorship. Voice or no
voice, the people can always be brought to the bidding of the leaders.
That is easy. All you have to do is tell them they are being attacked,
and denounce the pacifists for lack of patriotism, and exposing the
country to greater danger.”

– – Hermann Goering

Posted by Bob Hearn.

View of SF from the Marin Headlands

After our day of enjoying fine art, we decided to go out and get some delicious Thai food from Thai Spice which is located just south of Golden Gate Park, and then went to the Marin Headlands for a bit of a drive near Point Bonito. The remnants of many old gun emplacements are still visible here. I saw some views of the Golden Gate that I hadn’t seen before, and snapped a bunch of pictures with my cheapy $50 digital camera. When I got home, I stitched them together to make the panorama above. Not the highest quality, but a pleasant view nonetheless.

DaVinci and the Splendor of Poland

Carmen and I went to the Legion of Honor Museum in San Francisco today to get away from the war news coming out of Iraq and to get a chance to view a real DaVinci, his famous Woman with an Ermine, which is on loan from the Czartorski Museum in Krakow. There were also a number of very fine paintings, including some very nice views of Warsaw painted by Bernardo Bellotto which astounded both Carmen and I with their fine detail. It was a splendid display of Polish art, and well worth attending.

Scanning and Archival

I have a bunch of papers that are not valuable, but are rare and hard to find. Some I have only as faded Xeroxes, such as a copy of Anton Kutter’s treatise on Schiefspiegler telescopes, and Arnold Leonard’s work on Yolo telescopes. Others are in collections which are out of print, such as Tom Duff’s Polygon scan conversion by exact convolution. I’ve recently decided that I should scan papers that I think are interesting and preserve them in some digital form.

At the Hacker’s conference, Brewster Kahle touted the DjVu format as an alternative to pdf. For fun I decided to try to use my available tools to try to convert my bad xerox of Tom’s paper (15 pages) into a compact, reasonable form. I slopped all 15 pages into my super budget Canon scanner, and quickly converted them into 300dpi bilevel TIFF files. Each file was an 1,053,264 byte file.

The DjVu tools are open source, I got them by installing the djvulibre package in FreeBSD. The program cjb2 provides bilevel image compression of PBM files, so I made a little script that converted each page into a pbm, and then to a .djvu file. I specified that the compression could be lossy and that it should remove flecks, and then assembled them together using the djvm program.

The resulting file was 279,530, for all 15 pages.

That wasn’t quite good enough though, I decided to go ahead and use the online any2djvu server to perform OCR on the djvu file and stash it back inside. The resulting file with OCR is 313,192 bytes, and can be searched.

I then tried to make a pdf file out of it. I converted the djvu file into PostScript (using the djvups program) and then used ghostscript to convert it to a pdf file. The result wasn’t pretty: the PostScript file was 4,268,709 bytes and the resulting PDF file was 3,184,680 bytes, a 10 increase over the DjVu file. I have no doubt that Adobe Distiller could do a better job, but then, I don’t have Adobe Distiller.

Anyway, I thought it was a fun experiment. You can have a peek at the resulting DjVu file if you like. You can either install the viewer or if you have a Windows machine, install the plugin.

Seven Signs of Bogus Science

I recently spotted this article
on the subject of spotting bogus science. It’s enormously easy to be hoodwinked by science, and these seven rules can help you spot the quacks from real science.

  1. The discoverer pitches the claim directly to the media.

  2. The discoverer says that a powerful establishment is trying to suppress his or her work.

  3. The scientific effect involved is always at the very limit of detection.

  4. Evidence for the discovery is anectdotal.

  5. The discoverer says a belief is credible because it has endured for centuries.

  6. The discoverer has worked in isolation.

  7. The discoverer must propose new laws of nature to explain an observation.

A nice concise ruleset, one which matches various Internet loons with a high degree of accuracy.

But then I’m just part of the scientific establishment working to keep ’em down.


Just for kicks, I tried creating a simple shockwave animation.

I generated this with ming via php. Bindings are
also available for Python and Ruby.

Okay, it’s pretty boring. But if it works, perhaps something better will be forthcoming.

FPGA Miscelleny

This is just a potpourri of FPGA related items.
I’m fairly interested in hardware design (although have not really done very much of it). I’ve been looking for some cheap programmable hardware, and found out about Digilent. They have boards based upon the Xilinx Spartan II FPGAs for around $100, and the Coolrunner CPLD for only $39. Xess Corporation has more boards, for a bit more buckage. You can also find cheap board layouts for a Spartan II evaluation board if you are more comfortable having boards fabbed (this one is only a 2 layer board, should be fairly economical to have manufactured). Some people are interested in using FPGAs for audio applications or for central processing units of your own design.

OLED displays…

The new up and coming display technology is Organic Light Emitting Diodes, or OLED displays. OLEDs will be cheaper to manufacture, have a much faster response time, don’t need to be backlit, draw less power, are brighter, and can be viewed over a much larger viewing angle. In short, they are very nearly the ideal display device. We’ve had a prototype here at Pixar for a while, and it is impressive. Kodak is the first vendor I know to announce a product with an OLED display, although I suspect that every cell phone in the universe will acquire these displays in the next year, with computer monitors following closely behind.

I can’t wait.

What are they teaching our children?

Allrighty, I know, if I am going to get irritated by the ignorance of human beings, I shouldn’t read Slashdot. A recent article detailed China’s desire to
mine the moon for minerals
. Never mind that it is absurd (can we name a single element so precious that it would justify the cost of rocket launches to bring it back from the moon. I would have expected someone to bring up that. But no, instead we get gems like the following.

Any of those more versed in physics than myself care to comment on what lowering the mass of the moon could do? I am sure not enough would be mined to raise the mass of the earth enough to cause problems, but wouldn’t a great enough reduction in lunar mass decrease the force of gravity between the earth and the moon, thus (possibely) destabilizing the orbit?

Sigh. Or how ’bout:

Presumably when they talk about “mining the moon” they are talking about going there to mine Helium 3. This is an isotope of helium which, if available in abundance, would be a perfect fuel for clean fusion power generation.

Except of course that nobody has built a working, controllable fusion reactor.

Economically this just doesn’t make sense. It’s hard to imagine the level of technology to make it make sense. Even the most difficult to mine natural resources of this planet will be cheaper to recover
than any resource from the moon or asteroids. Lunar mining is a pipe dream.

Bizarre Fish

I love to watch nature documentaries and the like on PBS, but most of them go over information that has been rehashed a million times. But just when you get complacent, you realize that there are still things out there which are big and yet largely unknown. One of the most unusual is the Oarfish, Regalecus glesne. Besides being huge (reported lengths of 17 meters, verified lengths of 8 meters), the oarfish has a beautiful silvery body and a long thin shape which is very odd.
It has been photographed by Jonathan Bird, and also by Navy divers (who caught it on video!).
I also recall seeing video of a barely alive specimen that washed up on the beach, but wasn’t able to find it online.