Category Archives: Blog

Academic Papers Artwork baby names Blog blogging democracy Design ethics Facebook firefox Flickr folksonomies Google Google Docs Google Spreadsheets how-to information-architecture information-retrieval information design internet iphone journalism listserv mailing list maps mass media Online News Papers Photography plugin poll social-bookmarking social networking social software spam tagging trust Twitter Usability web-development Web2.0 webspam web standards WordPress Writing

House hunting the geek way, Part 2: Data-driven maps in Photoshop

In part 1 we created a simple heat map in Photoshop to figure out which neighborhoods would be good places to look for a new house. But distance from work and school isn’t the only factor worth considering. We can always add more radial gradients to show proximity to favorite restaurants, family members, and the like. But that’s really just more of the same.

Think about the things that make a neighborhood a pleasant place to be – low crime, low pollution, parks nearby, friendly neighbors – some of those things can be quantified and mapped. We’ll have to wait for demographers to release official neighborhood friendliness metrics after the next census, but let’s see if we can find some of the other data.

Step 3: Highlight on-map elements

At least one of the new factors we want to look at is already available on our map – parks. All the parks on the map are in one of two shades of green. Use the Magic Wand Tool to select park areas and then Select -> Similar. You can see how I’ve selected the parks in the example below.

megamap-example-parks

Now we’re going to do something similar to the concentric circles in step 2. Choose Select -> Modify -> Expand. You might have to play around with the number of pixels you expand by – for the scale I was working at, 20 pixels looked like close walking distance. Now use the fill tool with a low opacity to fill the area with the same color you used for the circles.

You can then repeat the expand and fill steps as many times as you like to build a heat map of park proximity. Don’t forget to change the blending mode to Multiply to match your other layers.

megamap-example-parks-heatm

You can follow similar steps for other on-map elements, like shopping centers, college campuses, bodies of water – it all depends on what you like to be near and what’s available on your base map.

Step 4 – Pulling in data maps

First, a disclaimer: this isn’t a tutorial on how to automatically pull data from a server and have Photoshop map it for you (but keep watching my blog for a similar project in the future). Instead, we’re going to pull data maps from other places on the web and fit them over our heatmap.

The hardest part of this next step is finding the maps. The number and quality of maps available depends on your location, but in general the best two places to look are county and city websites and nearby colleges. If you don’t find what you’re looking for under “Maps” try looking for “GIS,” planning departments, or property information. Also, many government web sites have poor search systems – try doing a Google search with the site operator instead. For example, a search for Cuyahoga County might look like this: site:cuyahogacounty.us maps gis.

For this example, I’m going to grab a map from Case Western Reserve University’s NEO CANDO site. Another good source for the Cleveland area is the the Cuyahoga County Brownfields GIS server. My wife and I both have graduate degrees and we really value education – so I’m going to grab a map of the percentage of people with bachelor degrees or higher by census tract.

Cuyahoga_NEOCANDO32443568931

Now that we have a data map, we need to clean it up a bit and add it to our base map. Open the data map in Photoshop and use the Magic Wand tool to select the black and gray areas – the lines and numbers. Use Select-> Similar to make sure uoi have most of it selected and hit Delete. Now Select All, Copy and Paste it into your map as a new layer.

You’ll might want to use the Magic Wand and Select-> Similar again to clear out all the white area around the map and leave it transparent, but you don’t have to – you’re going to change the layer blending mode to Overlay like the other layers anyway. At this point, I can almost guarantee that the data map will be much smaller than your base map. Chose Edit -> Transform -> Scale to stretch it to fit. There’s no sure-fire way to do this, just keep stretching until you have a good fit to known boundaries like coastlines and major streets.

Here’s the result:

megamap-example-college

Step 5 – Bring it all together

Now that we have all these different layers, it’s time to pull them all together in one heat map.  You have a few options on how to do this.  If you make all the layer visible at the same time your going to get a lot of very blue areas.  Instead, try lowering the opacity of each layer based on who important it is to you.  You can see an example of my Cleveland area map below.

megamap-example-final

If you want to make the strongest areas of the heat map more visible, start by making your base map invisible while leaving all your other layers up.  Go to Select -> color range and clikc the eye dropper on the darkest blue area you can find.  Now increase the Fuzziness until it looks like the best areas are selected.  Hit the OK button, create a new blank layer, turn off the rest of your layers, and fill the selection with your blue.  You can see the result below.

megamap-example-final2

Hopefully this has been helpful.  You don’t have to make your map quite as involved as mine, and of course if you are looking in a smaller area you can constrain your map further.

Stay tuned for more updates on this topic.  If you have a feed reader you can subscribe to my blog and if you’d like you can get email updates, too.

House hunting the geek way, Part 1: Using Photoshop to make heat maps

If you’ve ever moved to a new city and looked for a house or apartment you know how difficult it can be.  What neighborhood, which side of town?  Can we live close to my wife’s workplace and not to far from mine?

I thought I would share the method I used to find our last house, using Photoshop to build a heat map of the city.  Note that this is NOT the method I used to find our current apartment – watch this space for more news on that coming up.

Step 1 – Build a map

In order to build our heat map you’ll need a base map to place everything on.  Back in 2004 when I did this project Mapquest was still the best thing going, so that’s what I used.  If I were doing it now, I would go with Google Maps.

This is the most tedious step, since you’ll need to center your map, take a screenshot, then cut the map portion of the screenshot and paste it into your working image.  If you have a scanner and a nice print map you’d like to use instead, feel free to go that route.

You can see my example, a map for the Greater Cleveland area, below.  Click to see a larger version.  The inset shows you the level of street detail I found best – zoomed in close enough to see all the streets, but not so close as to make your map unusably large.

megamap-example-plain

Step 2 – Place your main locations

What are the three most important factors in real estate?  Location, location, location.  In our case we want to live close to the locations we need to go to on a regular basis.  For us that was two workplaces and two universities.

Heat maps are a great way to visualize information.  They are a perfectly appropriate choice for map location and distance information.  So create a new layer in Photoshop.  Choose the gradient tool and make sure you’re using a Radial Gradient.  The gradient should go from a solid color (I chose blue) to transparent.  Using the map, create a radial gradient about as wide as you would like to drive.

These smooth gradients can make it hard to make distinctions when you are zoomed in and, on a large map, will take up a lot of disk space.  So an alternative method would be to create a series of coencentric circles, each smaller than the last.  That’s the method I used in the example below.

megamap-example-locations

Once you have one good circle layer, copy it for each of the locations you want on your map and drag them in to place.  You’re probably going to want to change the blending mode for the layers so that you can still see map details – I recommend using Multiply and lowering the opacity just a bit.

In my example map, you can already see how this could help narrow down which neighborhoods to look in.  It also shows quite visually that there’s no point in trying to live closer to Kent – it doesn’t intersect with any of the other hot spots.

In part 2, we’ll take a look at pulling in data maps for things like crime statistics , highlighting other map features, and pulling it all together.  Also, I’ll have an exciting announcement about another project I’ve been working on soon as well.  Stay tuned.

Fixing a ‘This site may harm your computer’ warning, part 3: Clearing a spammed forum

Sun setting behind a sculpture in the park near Google Earlier I wrote about the steps you should take if your site has been hacked and is being slapped with a “This site may harm your computer” label. In that post we covered some of the sneaky ways scammers will insert text into your posts on WordPress and other blog software.

But what if it’s even worse? Let’s say you installed a forum like phpBB to play around with but haven’t been keeping up with security updates. Or, even worse, your ftp account has been compromised and spammers have installed their own bulletin board or other content in a subfolder or subdomain. You don’t want Google and Yahoo thinking you are a spammer, so what do you do?

In that worst-case scenario, you’ll first need to change your passwords and make sure you have control of any and all ftp accounts, telnet accounts, etc. You may need to work with your host to make sure everything is locked down. Web server security is a big topic in it’s own right so from here on out we’ll assume you’ve already got that covered.

Step 1 – Delete the spam!

The first thing to do is delete the spammy bulletin board. Go ahead and delete all the contents of the directory. Don’t delete the directory itself quite yet. This does two things – it stops the spammers from getting any benefit from wayward visitors to your site and it causes your web server to start serving 404s (not found) to search engine spiders.

You can go one step further and explicitly tell browsers and spiders that this stuff is gone forever- by serving a 410 (gone). You can do this with any server-side language, my example will be in PHP. Create a new index.php file in your formerly-spammed directory that looks like this:

<?php header("HTTP/1.1 410 Gone");
header("Status: 410 Gone");?>

This will cover the main directory and then you can use mod_rewrite to redirect all the deleted pages to your 410 file.

Step 2 – Update your robots.txt

At this point search engine spiders will be able to figure out that the pages should be removed from their indexes, but only one page at a time as they re-crawl your site. You want it out of there ASAP, so create a robots.txt entry to tell spiders to stay away from the whole directory. It should look something like this:

User-agent: *
Disallow: /forum/

If the spam was in a subdomain, you’ll need to make sure you have a robots.txt file in the root directory of the subdomain that disallows the whole thing:

User-agent: *
Disallow: /

Step 3 – Tell Google about the spam

Log in to Google Webmaster Tools and look under Tools -> Remove URLs.  Create a new removal request for the subdirectory or subdomain you’ve cleaned.  This might seem a little redundant, since you’ve already done two steps that will let search engines know you’re no longer serving up spam.  But it’s worth being as explicit as possible to get your site’s reputation cleared as quickly as possible.

Bonus tip:  Subdomains and Google Webmaster Tools

If your spammed forum was in a subdomain, let’s say http://forum.exmaple.com, you’ll need to add the subdomain as a new site in Google Webmaster Tools.  You’ll need to go through the site verification process for the subdomain, too – it won’t verify automtically like if you had added a subdirectory as a new site.

By the way, if you’d like some more tips about keeping your site clean and tidy, check out this great post on the Google Webmaster Central Blog.

Any questions? Comments?  Tips that I’ve missed?  Please post in the comments section below.