I Don't Want To Be Your Friend

The music blogs are ablaze with this new video by Radiohead. It's hypnotizing. House of Cards is among my favorite tracks on the album. Also note that there is a "making of" here and some interesting background on the data visualization technique here (hosted by code.google no less)

Open Innovation: New Corporate R&D Strategies

A short article on how Microsoft, IBM, and HP are shifting research strategies.

Pundits in recent years have taken to bemoaning a retreat by U.S. industry from basic research in science and technology. And indeed, it's easy to cite research labs whose glory days have come and gone -- Bell Laboratories comes to mind. But consider this: IBM, Microsoft and Hewlett-Packard together spend $17 billion annually on research and development.

from: The new face of R&D: What's cooking at IBM, HP and Microsoft

Auto-Generate KML C# Classes

Using the xsd.exe tool (in a default installation, you can find it here: C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\Bin), you can auto-generate KML C# classes. Make sure you have ogckm122.xsd (here), atom-author-link.xsd (here), and xAL.xsd (here) in the directory where you are executing the xsd command.

>xsd /c ogckml22.xsd atom -author-link.xsd xAL.xsd

This will generate a C# file called ogckml22_atom-author-link_xAL.cs.

Thanks to these two blog posts for the help: xsd command and multiple xsd files and Error generating classes for schema 'MySchema.xsd'.

Google Maps Street View in Middle America

My sister's family lives in a small Nebraskan town with an estimated population of 3,000 people and an estimated median house value of $127,000 (was less than $100,000 in 2000). Nonetheless, Google has taken its mapping technology to these streets and captured my brother-in-law and niece on film (without their permission). It appears Google truly does want a visual index of all streets in America. Indeed, according to Wikipedia, "Google has stated that its ultimate goal is to provide street views of the entire world, although the company has not disclosed in advance the exact dates when any particular locations will be added"

Now, there have been plenty of write-ups online about the chilling implications of this technology from a privacy standpoint (see end of this post). Admittedly, I had not paid much attention to said implications until now. What I find startling here is not just that my family was captured on film and automatically uploaded by Google but that this picture was taken while my brother-in-law was on his private property in front of his own house (with my little niece no less). Further, if you click on the "Full-screen" link, the Street View photo itself increases in size, which is even more unsettling. It just makes one feel invaded. Note, here, however that there is evidence that Google is running automatic face detection algorithms and blurring faces. Automatic face detection is pretty much considered a solved problem in the computer vision world (face detection is not the same thing as face recognition). It's difficult to tell if Google is in fact explicitly obscuring the face here or if the overall photo quality is just poor.


Although the photo quality published online is rather low, I would not be surprised if the original captured image was of much higher quality, which likely still resides in a Google database somewhere. I would imagine that Google has plans to apply computer vision algorithms for automatic object detection, 3D-routing, sign reading, sidewalk routing (for people routing, rather than car routing), etc. Thus, there would be little reason for them to invest the money into Street View and not capture the highest quality imagery possible.

Some related links
Wikipedia's Google Street View page has a Privacy issues section as well as a long list of links to articles and news items related to Street View's growth, governmental concerns, and privacy implications:

Privacy advocates have objected to this Google feature, pointing to views found to show men leaving strip clubs, protesters at an abortion clinic, sunbathers in bikinis, cottagers at public parks, parents hitting their children, males picking up prostitutes and other activities, as well as people engaging in activities visible from public property in which they do not wish to be seen publicly.^[19] Google maintains that the photos were taken from public property. Before launching the service, Google removed photos of domestic violence shelters, and allows users to flag inappropriate or sensitive imagery for Google to review and remove.^[20] When the service was first launched, the process for requesting that an image be removed was not trivial;^[21] however, Google has since changed its policy to make removal more straightforward.^[22] Images of potential break-ins, sunbathers and individuals entering adult bookstores have, for example, remained active and these images have been widely republished.^[23]

Mashable has a Top 15 Google Street View Sightings page including pictures of a possible crime in progress, female sunbathers, and men coming/going from adult related entertainment venues. Some more Street View oddities are listed on this blog post. The New York Times has an article entitled Google Zooms In Too Close for Some, which is also worth a look as it briefly explores the idea of public vs. private and ownership of data.

Nokia Morph Concept

The new nano?

Querying Big Datasets with MySql and Java

I received this error when calling the executeQuery() method on a PreparedStatement object. You'll note that within the JDBC MySql code, there is a method invoked called "readAllResults." It appears that the default behavior of the executeQuery method is to read all data records into memory before returning! This simply will not do when dealing with large datasets.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1444)
...
at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1746)
...
at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:1885)

The solution was non-intuitive but an easy find with Google. From MySQL JDBC Memory Usage on Large ResultSet:

If you are working with ResultSets that have a large number of rows or large values, and can not allocate heap space in your JVM for the memory required, you can tell the driver to stream the results back one row at a time.

To enable this functionality, you need to create a Statement instance in the following manner:

stmt = conn.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY,
         java.sql.ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(Integer.MIN_VALUE);

The combination of a forward-only, read-only result set, with a fetch size of Integer.MIN_VALUE serves as a signal to the driver to stream result sets row-by-row. After this any result sets created with the statement will be retrieved row-by-row.

There are a few caveats, which are described in the following links:

• MySQL JDBC Memory Usage on Large ResultSet
• Reading row-by-row into Java from MySQL
• JDBC API Implementation Notes
• MySql Streaming ResultSet
  

Row Limit You Ask?

As much as I love Excel 2007 (finally an easy way to save your graphs into templates!), I cannot believe they still enforce a row limit. A "row limit" you ask? Surely not, I mean, it is after all 2007--the years of mandating artificial limits of data sizes are long over, no? Unfortunately not. Excel 2007, row limit=1,048,576.

Um, why?

I'd like to think it's because Excel's dev lead programmed int [] rows = new int[1048576] at the top of the Excel document model.

MyExperience Website

I finally updated the MyExperience website such that it now actually has real content. Although the MyExperience source code has been available for quite a while now at http://www.sourceforge.net/projects/myexperience, the website http://myexperience.sourceforge.net was mainly a skeleton page with broken links. This has been fixed :)

Note that the SVN activity log on the leftside of the webpage is actually live. I wrote a php script to dump the two most recent commits to the frontpage.

Update 10/02/2007: I was alerted to the fact that http://www.sourceforge.net/projects/myexperience was a broken link before. I had accidentally left out the "projects" part. Now fixed.

Art Through Technology

Any guesses as to what this is?

I Said, "No Negative Values"

I selected "Logarithmic scale" on an Excel 2007 graph that contained negative data points (which I didn't realize until it was too late). I was told in a not-so-subtle manner that this is a bad thing. Needless to say, I was graphing about 12,000 data points--only about 500 of which were negative. I would guess that this is also the number of error dialogs that popped up.

Should I select the "Was this information helpful?" link? Microsoft certainly seems to think so. I guess they're not really following the adage: less is more.

I'm going to start clicking OK on these babies--probably be done in about an hour.

Charts that Flow

BoingBoing has been linking to lots of flowcharts recently, see Criteria for proper tactical usage of phrase "Oh, Snap!" and Medieval sexual decisionmaking for penitentials . My favorite, however, is this one from Toothpaste for Dinner: the Panflute flowchart. Comedic genius.

Website Launch

My personal website is now officially live. I tested it in Opera (Mac & PC), FireFox (Mac & PC), Safari (Mac), and IE7 (PC). There was a slight JavaScript bug in IE7 that I resolved today thanks to webdeveloper.com (see my post here). It turns out that IE7 does not handle <script .../> correctly. Instead, you must use the longhand closure <script .../></script>.

There are a few small rendering discrepancies across browsers, particularly in IE7 vs. everything else but I'm willing to live with them for now.

You may notice the dynamic content in the right side column of the about me page. This was done using a mixture of php and JavaScript. I plan to post those scripts on the website (in the hacks section) for public consumption. This was my first experience using php and I was impressed. It's amazingly easy to parse XML data and the library calls are straightforward. I imagine that if I did something more complicated, I would definitely need some sort of php debugger.

Website Header Reflected

From Mockup to Website

I've been using PowerPoint to mockup interfaces since I got into graduate school. This weekend, my goal was to implement my website ppt mockup in actual html (and learn php and CSS in the process).

Here's the ppt mockup:


Here's a screen capture of the actual site rendered in FireFox:


Time will tell whether this renders correctly in all browsers, FireFox and Opera seem to do fine.

Running SSH Server on Windows XP

This howto document created by the University of Waterloo on "how to setup the Cygin SSH Daemon on Windows 2003" is fantastic. Note that every piece of advice translated directly for me to Windows XP Professional (except for the keygen stuff which I don't use).

There was one thing missing, however, and that is to unblock port 22 in your firewall. By default, the Windows Firewall blocks this port. If you don't, the ssh server will be inaccessible from outside your domain. :)

Easy Way To Remove Folder Structure in Windows

I'm in the process of switching from CVS to SVN. I wanted to remove all the underlying "CVS" directories for my projects--I first thought to write a recursive batch file that would search and destroy all CVS folders. Frankly, however, I did not really want to spend time looking up old DOS batch commands. J. Lester and T. Scott had the idea to use Windows Search functionality (yes, I know, hard to believe it's good for something). Basically, (1) open the folder that you want to recursively delete some files/folders in, (2) then open the Search box, enter your search criteria, (3) select files and delete. It's that easy; see below:

(1) Open folder of interest

(2) Search

(3) Select and delete

Be Jubilant: Rumsfeld Resigns

Worse than Robert McNamara? Time will tell...

(link)

Web 2.0 (that I find useful)

• Wikispaces : free wiki hosting
• Backpack : free file, image, calendaring, todo list hosting (pages can be marked private)
  
• Farecast : airline fare predictor
  
• Zillow : real estate pricing predictor
  
• Digg : social bookmarking/linking site
  
• Google Calendar : online calendaring
• Del.icio.us : social bookmarking
  

Inkscape Rocks

Hard to believe something so good could be free. Go check it out yourself here.

Iowa State's Cave

It's fully immersive you know. 6-sided even. And it's only getting better. (link)