I run the text-processing NLTK demos & NLP APIs on Linode, which has been a great VPS provider. Despite the fact that the memory of all the trained objects far exceeds the available RAM of the instance, the demos and APIs continue to run quite well.
StreamHacker is hosted on Just Host, and the only problem I've ever had was a minor WordPress hack by a Hamas support group. But that wasn't Just Host's fault, and their support response was fast and helpful. When web hosting is a commodity, providers must compete on price and customer service, and Just Host does a great job on both fronts.
When you need on-demand computing resources, the Rackspace Cloud is great. I like their instance options a lot more than EC2, and the servers always feel very responsive. Even the smallest instances have access to 4 CPU cores. Their prices may be a bit higher, but it's worth it to get that extra level of quality service.
Clicky is by far the best web analytics provider I've ever used. They truly care about user interface and information design. The yearly fee is totally worth it when you compare them to the free alternatives - no other UI I've seen is even close to as good as Clicky's. Plus, Clicky lets you add Feedburner stats and track saved twitter searches, which I find quite useful.
If you need custom analytics, Mixpanel is a great provider. Unlike standard web analytics, with Mixpanel you can track whatever you want. I use it to track which parameters and trained objects people are using for the text-processing NLTK demos & NLP APIs. Since the demos and APIs are all POST requests, tracking parameters with standard web analytics is impossible. But with Mixpanel, it's trivial.
Whenever I'm working on a consulting project, I use FreshBooks for time tracking & invoicing. Their timer utility is quite useful, and makes it really easy to track your time and what you worked on. These time entries can be then be transformed into a pretty PDF invoice with just a few clicks. Paypal setup is a breeze, and their free plan gives you plenty to get started.
I buy all my domains (probably too many through namecheap. They're cheap (as the name implies), reliable (unlike some more well-known competitors), and have actually put some work into creating a decent UI. In addition to domains, they also provide SSL certificates and basic web hosting.
Microsoft Natural Ergo Keyboard 4000
After getting the Microsoft Natural Ergo Keyboard 4000, I'm not sure I'll ever use anything else. My wrists had been hurting using standard keyboards, and I feared I was developing carpal tunnel syndrome, but this keyboard changed all that. In my experience, Microsoft products tend to increase pain, but the Microsoft Natural Keyboard has done the opposite.
Linksys WRT54GL Wireless Router
The Linksys WRT54GL is a great wireless router, and it gets even better when you install DD-WRT on it. I installed DD-WRT on my router in order to connect non-wireless desktops & servers to a wireless network without any add-in cards. It works great, and unlike many routers, it'll run for months, maybe years, without requiring a reboot.