This beta has been three years in the making. We’ve been working on Luster for that long. There’s plenty more to do, and we definitely want to hear suggestions from everyone. Let us know what you think.

We were thrilled about the article, and have been absolutely swamped with emails and calls about it since then (hence the late post). We loved working the D&C on it as they were really nice and a fun group of people to work with. It seems we now have some expectations of us to fulfill.

We’ve been working with our client on this medical imaging system for over a year now. Our part (the 3D part, of course) is one element in a larger application. The entire system is developing very nicely. Here is the latest version of our 3D renderer for CT and MRI scans. What you see above is the most basic rendering: direct volume rendering with no coloring applied. Below is the same scan rendered from above. Keep in mind that the scan is rendered at interactive framerates, so the user can change all display settings and navigate around and in the scan in real time.

We’ve added in another rendering mode which can be useful for some scans: iso-surface rendering. This mode creates a more “solid” image of the data and allows us to do a few different kinds of effects compared to the direct volume rendering mode.

Notice the high-quality lighting. There are numerous lighting and coloring settings available for isosurface mode. Many of the same data manipulation controls are available in this mode as well. Here’s the same scan but we’ve now focused on on the pelvis and hip bones. You can see that the scan has a lot of data in it and that once we start exploring the inside of the body it gets quite jumbled. Don’t worry, a radiologist knows what they are looking at. Also notice a few lighting artifacts (bands and stairs) because even though this data has some better resolution than previous scans, it is still relatively low. For this entire abdomen there are only about 160 slices.

I do not know who the original author is. It was uploaded here.

I’m referring to the data collected during medical scans. There are many different kinds: PET, CT, MRI, Ultrasound, etc. I’m dealing mainly with CT and MRI and the data sizes from these scans is exploding. This is a result of high-quality scanning and collection techniques. This should be viewed as a good thing. These machines cost a lot of money, but they can be used over and over. Higher quality scans produce better images to analyze, which in turn are easier to use as tools for diagnostics, recovery tracking, clinical trials, surgery planning, and probably a lot more.

What kind of sizes am I talking about? With the newest machines we are talking about Gigabytes of data per scan. Often in one sitting multiple scans will be carried out. For instance, many times, a scan is taken before and twice while your body is filtering a contrast chemical. That means modern machines can produce several Gigabytes of data per patient, per visit. How do we store this? How do we process it?

I don’t know about storage issues. I’m hope Radiology practices are solving this issue. I certainly hope they don’t just throw away your scan when you leave their office. My focus is on the processing. Currently Radiologists look at these scans as a series of 2D slices. With modern machines able to produce thousands of slices per scan this method is going to be very outdated very quickly. Instead we can use modern consumer graphics hardware (the video card in a normal computer) to reconstruct this scan in full 3D. This lets the Radiologist view the entire scan at once and interact with it in real time.

The data overload is an enormous barrier to this type of application. A modern scan will not only overflow the video card’s memory, but before long it could begin to overflow the computer’s main system memory. Managing this flood of data and feeding it into the graphics pipeline in a sensible way is exactly what we are tackling right now. It’s a critical problem, because we want Radiologists to be as effective as possible with this new level of detail.

I will post again when we have a more complete prototype of our new 3D Medical Scan Visualization software.

RSS Feeds geo-located on the earth

You can browse the news stories by hovering over the little pins sticking out of the earth. The information will pop-up in a tooltip. This gives you the title of the story, its source, category, and a brief description/summary if it is available.

Browsing an news story's information

As you explore the earth, blue pins will start to pop in around you. These blue pins are discovered geo-located Flickr images. The application crawls through Flickr, finding those images that have been geo-tagged. It then populate the area around where your looking with those images. If you roll over those pins, a thumbnail of the image dynamically downloaded from Flickr will float beside your cursor.

Viewing the thumbnail of a geo-located Flickr image

Nighttime across the US overlayed with its new stories

By clicking the blue pins, the full-resolution image will be downloaded from Flickr and shown in a pop-up.

The high-res image of a geo-located Flickr image

You can dynamically filter the data by checking RSS feeds or Flickr on or off. In the future you will be able to filter both news and images by category or tag. In the image below the RSS feed has been turned off, leaving just the blue Flickr pins.

RSS feeds turned off, just viewing Flickr images

Pure globe with all geo-located data flitered off

With Luster we can take advantage of available web services and visualize the location of data in real-time.

We can take information from online services and databases and plot them onto the globe. Viewing information by its location is just one way of exploring it, but done correctly that can be a powerful way to visualize and see patterns in the data. What you see above are the news stories from Yahoo’s Top Stories RSS Feed. The application checks for changes every minute, so you have a real-time view of Yahoo’s top stories as they happen across the world.

And since RSS provides us with all sorts of useful information, we can let that information that is attached to each news item be viewable in the application. You can roll over each pin and see the story, its source, description, etc. When the user is interested in reading the story, a simple mouse click will navigate their browser to the story’s link.

Really, RSS feeds are the just the beginning. More data is being geo-tagged online all the time. Flickr has some 35,000 images with location tags and a powerful API that can access them. Twittr recently announced that they will be letting users opt-in to have their tweets geo-tagged. If a significant number of users start tagging their tweets with this information, we can use our application to plot where tweets happen across the world in real-time.

For a more advanced demo, we can allow the user to interact with different parts, or perform actions on the system. For instance, for a car we could open the door and move the camera in to peer into the interior. We can let the user look under the hood and rev the engine (play a recorded sound of the actual engine). You can see in this video below an automated exploded view which we can do for any other products. We could also attach information to the product, or its parts that the user can easily access. A car might have information about optional extras, with links that lead out to the manufacturer’s website with even more information.

These sorts of application makes exploring a product more fun, and easier to navigate. Here’s some media from the demo.

For those versed in medical visualization you can note that there is no special transfer function used here. The pixels are colored directly with the density value read from the scan, which means the 3D reconstruction matches the 2D slice view very closely.