Sluggish Mac OS X Performance Resolved

I’ve been having problems with horrible performance on my Mac desktop.  Constant grinding of the hard driver, non-stop lagging apps.  Turns out MDS (the spotlight process) seemed to be constantly running.  Google brought some suggestions such as completely rebuilding the spotlight index, but this ultimately didn’t work.  Even turning spotlight off seemed to miss the mark.Finally, I made two changes which solved the problem completely:

  • I added /Library/Backblaze to the Spotlight exclusions list.
  • I forced a Time Machine backup, then added that volume to the exclusions list.

That’s it!  A simple fix restored the performance of my machine.  I’m unsure as to why it started happening all of a sudden, but I’ll take it.UpUpdate 2/5/2011: A clue to what might be going on as evidenced by the following command:

  • sudo fs_usage -w -f filesys mdworker

This command shows exactly what Spotlight is doing, and essentially it appears that whenever a file is accessed on the filesystem, Spotlight gives it a look for indexing.  This means when you’re backing up (either via Backblaze or Time Machine) lots of files are getting “touched” and thus need to be examined by Spotlight.  This impacts an already I/O intensive process, bringing the system to a crawl.  Since this post, I’ve had zero performance problems and I’m going to consider this one solved.  Also, on my MacBook Air, Backblaze had already added itself to the exclusions list, so it’s possible that this has been addressed when you do a fresh install of Backblaze nowadays.

Google is Getting Copied! So What?

There’s been an uproar (at least in tech circles) over the past week about allegations from Google that Microsoft’s Bing search engine was copying results from Google’s offering.  There goes Microsoft again, being evil!  There goes Google again, being good!  At least, that’s the popular sentiment.  However, the whole thing struck me as bizarre.  Technology companies have for years copied each other’s offerings, putting minimal twists or variations in their own products:

  • Microsoft Windows copies Apple’s MacOS.
  • Lotus copies Visicalc (the first spreadsheet).
  • Google copies every search engine in existence when they start.
  • Google’s Android OS copies Apple’s iOS.

Now, most people would respond that this isn’t the sort of detailed copying that Google’s objecting to.  Macro-copying of a product or service is OK – that’s just old fashioned competition.But what about copying a particular feature?  That seems to be OK by Google as well:

These were both good ideas, and were major differentiators from a user’s perspective for Bing.  Bing’s attractive backgrounds before starting a search were a classy very non-Google approach to take that immediately set it apart from it’s main competition.  Google felt like copying it was not an issue at all.Macro copying of an entire service or product line is OK with Google.  Copying specific features is OK with Google.  Maybe Google has an issue with the specificity of Microsoft’s copying.  They don’t like the very precise manner with which Bing lifts its results.  Still, Google has done this before as well when they copied Yahoo’s marketing in an almost pixel perfect ripoff.Now, my example provided above is admittedly much smaller in scope and much easier to have happen accidentally by some ill-advised graphic design intern, but I think it’s ridiculous to post about Bing copying Google when it’s a fact that all search engines routinely spend significant time analyzing their competitors.  Google has plenty of research and monitoring directed towards Bing, Yahoo, and others, otherwise how would they have known Bing was swiping their results?  In other words, we’re back to the copying that Google is comfortable with – a little higher on the food chain, a little more abstracted.What Bing and Microsoft are doing is certainly dirty, and annoying.  But it’s also stupid, lazy, and incredibly shortsighted – they might be improving their results temporarily but now they’re struggling in a PR battle they’re sure to lose.  They’re also not working on improving their own rankings or understanding why Google’s results are what they are.  Still, I can’t help but feel Google’s righteous indignation is a little over-the-top on the eve of it releasing several major copied products: its Facebook killer, Honeycomb (the tablet version of Android), a Groupon killer (Update: A reader provided this link showing an almost perfect ripoff from their beta site), and many more.Iteration is part of competition, and the way to beat a competitor (especially one who copies a little too closely) is to keep iterating.  Google exposed Bing, but I can’t help but wonder if they should have just ignored the whole thing, chuckled to themselves, and kept working.  I wonder how many millions went over to Bing and tried it after this announcement.

Some Quick Thoughts on the Kindle

I got an iPad for Christmas.  For me, it was a relatively simple decision.  I fly/travel a lot and was burning a lot of space carting books around, especially newer hard cover books.  I also already use an iPhone 4G and have been extremely happy with the platform and device, so I was excited to see a lot of my favorite applications make their way to the iPad.  Essentially, I was looking for an eReader that had good battery life (at least 8 hours) which would provide flexibility to do other things.  This ruled out the Kindle eInk device and the Nook but I felt like the Kindle bookstore was more mature, had better selection, and was more portable (available on more devices).  All of my iPad reading is done with the Kindle application.The Kindle application ecosystem has gotten a lot of things right.

  • The highlighting is a killer feature.  Read a book, non-destructively highlight it, view your highlights on kindle.amazon.com.  This makes note taking SO much faster, easier, and portable.
  • The device syncing is fantastic.  I generally will read on the iPad most places but read on my phone if I’m biking at the gym.  The iPhone app syncs up where I left off and I don’t have to think about it.  Yes, this is a little feature and an obvious one, but it makes a big difference.
  • The book selection is very good.  Only when I’m trying to find more esoteric books is it a problem, and then I dutifully click “Tell the Publisher”.  Amazon should give you an email when it becomes available, but they don’t seem to do that right now.
  • The desktop applications are nice and consistent with the mobile applications.
  • The one-click buying is ridiculously convenient, and the books downloads almost instant.
  • Page turns are instant, and pleasant.
  • The built-in (and offline) dictionary is used a lot more than I thought it would be.

The Kindle software needs to improve on a few things:

  • I can’t find a good way to export all of my annotations in plain text.  I’m wondering if this is some sort of DRM policy to prevent people from highlighting the entire book (which I haven’t tried) then exporting it.  Anyway, I have to copy and paste my highlights directly from the web page right now which is not the end of the world, but is annoying.
  • It’s super annoying that the iOS Amazon application doesn’t include the Kindle eBook store.  You have to use your web browser to hit the store and purchase a book.  It literally does not exist within the mainline iOS Amazon app.  Search for a book there and it won’t show you if it’s available on the Kindle.
  • Their DRM policy is really stupid.  This is something everyone says to me when I mention I use a Kindle.  Almost all DRM complaints would go away if Amazon let you do an time unlimited lend of a book to another account which prevented you from reading that title while it’s lent out.  Currently they let you do one lend (total, ever, never to be lent to someone else after that one-time lend) for up to 14 days.  Stupid, stupid, stupid.  My response to those complaining about these restrictions is that it’s relatively simple to strip the DRM, and just like that, Amazon is only hurting it’s paying customers and providing an incentive not to buy and to pirate.
  • The Kindle app should start supporting every file format out there that’s available.  I know they’re starting to do this, but seriously, what’s the holdup?

Things I’m unsure about / haven’t tried yet.

  • It’s unclear to me how PDFs work.  I’ve got a bunch that I’d like to have for reference, etc., and locally manage (or access via Dropbox), and you can’t just drag a PDF into the app and have it show up on your device.  Very annoying.  I get that there’s some sort of post-processing that needs to happen for eInk devices, but it seems like this could all be easier.  Maybe you could post-process yourself on your desktop app and save everyone the trouble.
  • Haven’t done much note taking.  I’ve found note taking to be almost unnecessary when you can just highlight content and move on.

Parting ThoughtIf I was involved in an eInk company today, I’d be doing everything I could to bridge the “tactile gap” that still exists for most people.  The Wife has emphatically stated that reading for her is half tactile.  She likes the page turning.  The physical pages.  The weight of the book.  The smell.  She’s like an alcoholic who loves everything about the experience: the glass, the ice, the sound of the pour, the smell.  It’s easy to dismiss this as a triviality but it’s going to be a long-term battle for the next twenty years at least.  The company that can make an eInk page that feels like paper inside a book with pages to turn might have a shot at interesting these people.  Nice leather cover, plenty of pages (a thousand?) to accomodate 95% of books out there.  Pages to turn, etc.  It’d be like the book lover’s smokeless cigarette.Maybe I’m just out to lunch, but I’d love to see a double-blind study with a well-worn eInk “book” compared to a normal book and see if people could tell the difference or would care.

Tivo Premiere iPad App

I’ve been waiting for months for the Tivo iPad app.  I got emails saying it was coming out in November, shortly after buying my first Tivo, and when I got an iPad for Christmas, it was one of the first things I looked for in the App store.  Except it wasn’t there.  The Tivo iPad app spent several months prominently featured on Tivo.com and yet when you went to install it the words “Coming Soon” where there instead of “Install App.”I woke on Saturday and found out it had been released.  I downloaded it, installed it, connected it to the Tivo box just like the instructions said, and it seemed like it was working.  But the application couldn’t download information from the box.  I’ve got the Tivo wired directly into the router that’s broadcasting my wireless signal, and have managed to download entire episodes and other shows at decent speeds over that network link so I’m pretty sure that the network isn’t the problem.The Tivo box itself isn’t recording anything either, so it didn’t seem to be under load.  Possibly the most frustrating thing I noticed is that when I tried to use the remote function on the iPad, I could see the router connected to the unit blink exactly when I pressed the buttons (there’s nothing else active connected) and yet the unit wouldn’t respond.  Even the information which should be streaming in directly from the internet (things like actor bios and show information) loaded slowly and usually not at all.Midway through playing with it, my iPad lost its internet connection, which has literally never happened in my house, and even though I could join back up, I could never get it working well.  Last night I noticed that the App Store was signaling an upgrade was available for the application, so I downloaded it and when reading the change log I noticed the rather bizarre note: “Users who are experiencing slow performance should try turning off Bluetooth”.  For the life of me, I can’t figure out why this would matter, but I turned it off, and all of a sudden the application was usable, with fairly decent speed.How is the application?  I’d say it’s consistent with my opinion of the rest of the Tivo experience: OK, but a very long way from great.  My overwhelming impression of the entire Tivo experience is that it feels like the people who design their products don’t use their products.  Case in point: it’s extremely irritating that you can’t filter your channels on the box or the iPad application to show those channels that have HD equivalents AND those standard definition channels which don’t come in HD.  This is the case with almost any cable subscriber: certain channels like ESPNU just aren’t available in HD.  I don’t want to hide those, but if an HD channel exists, I want to use the HD channel.  Give me a filtered list that matches the usage pattern of most of your subscribers!The search experience on all devices (including Tivo.com) is completely horrid, so you end up paging through all your channels (you can’t just filter sports channels for example) at the prescribed time of your event, then finally get to ESPN.  Then you schedule it, but it’s not HD ESPN, because that’s fifteen pages down with the other HD channels.  On the web, they have a drop down which lets you pick t he HD channel quickly.  This feature is missing from the iPad, which makes it useless for scheduling recordings, just like the Tivo box itself.The remote works pretty well, but so does my physical remote.  A keyboard is nice for searching, but the search functionality is completely broken, so that’s not a very big win.  I don’t understand why they chose to not include a streaming function on the iPad app which would have made it a killer application – watch your recorded shows on your iPad, thus doubling your TVs.All in all, this app feels like a meaningless addon.  You can’t easily search or schedule HD shows.  There’s no filtering of events.  Sports season selection is still completely nonexistent (can’t schedule all games the Wolfpack play in, for example).  You can’t stream video to the device, and you can’t watch netflix or other web videos from the iPad.  The main benefit the iPad gives you is the ability to browse your Tivo and see your Todo list while someone else is watching.I’m afraid that Tivo is rapidly approaching irrelevance.  They have maybe a year or two left until someone at Boxee or XMBC or Apple builds a box that is Cablecard equipped and can search internet listings.  The complete lack of basic features (another example: on the program guide, you can’t tell if a program is scheduled to record, you have to go back to your Todo list) makes it a bad DVR.  Netflix and other internet streaming options are available on almost every other device.  Place shifting is available from Slingbox, and other devices like XMBC and Boxee and most Bluray players can stream local content via DLNA.This is how companies die, and it’s a shame because Tivo has had plenty of time to get all of those things perfected.

Christmas – a Time of Giving

Normally around Christmas people are focused on giving.  Giving to family, friends, maybe even coworkers.  Some give time to charities, some make donations to others.  One of my annual traditions for Christmas is giving to the Electronic Frontier Foundation (EFF), a digital liberties organization that fights for freedom of speech online, privacy, and other causes that are generally too technical to garner attention from other organizations.

This year I’m adding a very similar organization to my annual Christmas list, Software Freedom Law Center (Hi Aaron!),  which similarly advocates for issues that increasingly plague and impact our lives, whether we understand them or not.With all the buzz going on right now about Wikileaks maybe you’ve thought about these issues a little more than normal.

Depending on your understanding of the issue or political orientation, it might be hard to believe, but freedom of speech online is a relatively fragile thing.  Maybe it’s because technology appears to be finite and controllable, maybe it’s because online speech travels much faster and is easier to consume, but it seems that governments and governing bodies tend to naturally flock towards the idea of speech regulation of technical mediums.Groups like the Software Freedom Law Center and the Electronic Frontier Foundation work to preserve freedoms on our behalf.

When I say “freedom of speech online” or “freedom online”, we’re also talking about fighting against overly broad software patents, violators of the GPL, warrantless wiretapping of online communications, and a whole bunch of other issues that may seem only tangentially related to speech, but do impact your voice online.In addition to the work they do, I’ve had the privilege of interacting with and knowing a few individuals from both organizations, and they’re great people as well!  So Merry Christmas, and if you’re looking for a great gift idea or a cause to support, check out both groups.

Thinking Like a User: The Bank Analogy

Like every software organization, we have trouble getting our developers how to think like users.  It’s a time consuming process, and when you’re in an industry like healthcare, it can often be extremely difficult to visualize exactly how a product is going to be used or how a process might affect a user.  You’ve certainly never been to med school, or worked in a pharmacy.  These things are utterly foreign.  However, visualizing a user’s workflow or responsibilities is often the difference between a mediocre product and a great product.One method that I’ve found to be illuminating to our engineers is to restate every message, function, or process in terms of a bank, with them playing the user character.  Instead of the error message being “Purchase Order Alert: We’re sorry, but there were one or more items missing from your recent purchase, click here for more details“, rephrase it as “ATM Deposit Alert: We’re sorry, but there were one or more checks missing from your recent deposit, click here for more details.“Makes a difference doesn’t it?  Now our verbose, overly-polite alert text seems almost ridiculous.  Tell me what happened with my checks! Don’t make me click through for more detail!  Especially if I’m on a mobile device!  That’s my money and it’s important!“ATM: 1 check for $100.00 was returned.”That’s much better.  I feel notified and in control as a user, and it was short and sweet.  Very cellphone friendly.“Order 123: 1 item shorted: Tylenol 20MG”The point is that users actually use the products we build.  Things like error messages and process flows are important.  As developers, we often think of these things as control points, or logic trees, and don’t stop and relate what’s going on from a user’s standpoint to any of the important systems or applications that we ourselves use.  The bank analogy should help you get in the habit of stepping aside and thinking outside your code for a bit because it pulls the process or message into the realm of your experience.Even the process of constructing the analogy can be very instructive.  If you can’t quickly pull together an analogy that describes what you’re doing in terms of your own life, you probably have no clue what you’re doing.  In other words, you’re probably doing it wrong.Next time you’re solving a problem, try restating it using concepts you’re familiar with in your own life.  You might be surprised at how useful this technique can be.Updated: Sunday, 12/5/2010Today’s New York Times Magazine had an article on Jamie Dimon (I think it requires registration) that talked about his management and conversational style.  One of his favorite things to do was equate banking principles back to ordinary life.  Seems like we’ve got company on the other side of the fence too!

An Open Letter to Mint.com: Stop storing my bank credentials!

A lot has changed over the last few years.  It seems like forever ago, and yet, it was only in 2005 that AJAX sprang forth and ushered in the buzzword of Web 2.0.  And it’s great – rich applications that are delivered quickly and efficiently allow me to do things online that I never thought possible. And yet, there’s a dark side to the Web 2.0 craze for APIs and tools and importing and exporting data, and that is that we’ve taught our users to embrace man-in-the-middle attacks.  Every time I see a website asking me for my Facebook password I cringe, but nothing pales in comparison to the nightmare that is Mint.com.

I love Mint.com.  They have spectacular visual design, a great product, an entertaining and informative blog, and a great iPhone app.  I know tons of people who love Mint.com, and yet, when surveying my digital life with a critical eye, I know of no greater security risk than Mint.com.  It’s still astounding to me that Mint could grow from a small startup to being acquired by Intuit in the space of a few years and essentially retain unlimited liability by storing user’s logins and passwords to their entire financial lives.  Yikes. 

If I were turned to the dark side, I would immediately attempt to hit Mint for their millions of users credentials which provide me completely unfettered access to their accounts, most of which are not FDIC insured.  This means that when someone hacks Mint, they’ll be able to pull out all of my money, transfer it, etc., and I’ll be responsible because from the financial institution’s perspective they aren’t liable for me entrusting my credentials to a third party. Sure, Mint encrypts their password database, but somewhere that password is known or stored.  It has to be because they have to use my unencrypted credentials to login.  Sure, there are a bunch of ways they could monitor this access and mitigate risk, but at the end of the day there are usernames and passwords floating away.

There is simply no technical reason the financials institutions out there can’t work with Mint and every other API providers/consumers out there can’t implement an OAuth authentication solution.  For the nontechnical of us who are reading this, an OAuth solution is essentially a token based method of authentication.  A key based authentication mechanism doesn’t necessitate handing over your username and password to a third party, instead, you grant a key (and depending on the API, limited access) to Mint which can then login and grab the information you need.  If Mint gets compromised, your financial details might be stolen, but at least they can’t access the upstream account with the same type of access.  In fact, this is what I was really hoping would come out from the Intuit acquisition: for Quicken you used to have your financial institution give you a separate login or key for Quicken specifically. To be clear: this is originally the financial institution’s problem.  They should be providing OAuth based services for Mint and  others to consume.  However, this has now become Mint’s problem to address.  Also, hindsight is 20-20.  What may have started out as a great application for a developer to track his personal finances with an acceptable risk quotient has ballooned into one of the largest and best avenues for tracking finances in the world.

The simple fact is that today, when you change your financial institution credentials, Mint breaks, which means they’re scraping the content from financial institutions.  Financial institutions are in on it too – it should be easy to see that a large percentage of their traffic is coming from one domain.  Even those sites that use a two-factor “security questions” approach are accessed via Mint by saving all possible security questions!  Financial institutions could easily block Mint by adding CAPTCHAs to their login protocols, but since I personally know several users who have changed banks to use Mint, my guess is there’s sufficient pressure to maintain Mint’s access. Some might say that I’m being overly paranoid because we’re used to saving usernames and passwords on our local machines and while it is true that from a direct comparison perspective Mint probably has the security edge over my Macbook Pro, from a risk management perspective it’s quite a different story.  All of a sudden it pays to hire a team of evil programmers for a million bucks to gain access to Mint’s millions of users.  Consider too the fact that most people re-use a single username or password as much as possible – this means cracking a lower security database (a web forum, etc.) can leapfrog access into those same user’s Mint accounts.  The less we’re using usernames and passwords for services, the better. What’s the solution?  I think a three pronged approach should be considered by any modern technical service that holds data of value:

  • Institutions should provide rich APIs in the first place and aggressively prevent screen scraping.
  • APIs should clearly segregate between “read only” and “read and write” access levels.  Mint.com can “read” my financial data but can’t “write” and pull money out of my account for example.  API access could further be segmented to only allow access to pieces of data (e.g. financial sums only and not transactions, or both, etc.)
  • APIs should use account credentials for access, but instead should be key or token based.

This might sound complicated, but in practice it’s very straightforward.  I simply login to authorize a request made by an application (anyone authorized a Netflix device recently?) and that’s it. In an increasingly networked world, application service providers bear increased responsibility to provide safe computing to users.  The old standards of storing usernames and passwords within applications need to change to reflect a different risk model.  This means both providers (financial institutions) and consumers (Mint.com) of data.  I want to use Mint and recommend it to others so I’m hoping that they can bring their clout to bear and work things out with financial institutions to solve this problem.

The Non-Technical Tech Talk

Two weeks ago (was it three?) I spoke at an ACM sponsored Tech Talk at Purdue University.  My mission was to give perspective on what it takes to run a large and quickly growing software team.  Many startups (including some I’ve been affiliated with in the past) will grow from very small teams to departments of over a hundred in a short time.  That kind of growth is explosive so as I thought about what to say during the presentation, I was struck that very rarely do I hear much about the human element of running a software organization.  I never heard about it in school. Focus on the people side of software construction has been one of my major recurring management themes because, as I told the group of students at Purdue, software development is one of the most human-intensive, human-dependent disciplines/arts/crafts/industries that exist.

To many, this might exemplify a supreme irony, but this has always been a core belief of mine: you can’t build great software without great people.  You can’t build great software without great teams.  Hardware and tools are easy to come by now, and the end result is that in today’s software industry, people are the only variable that means anything.  That’s why software companies should be psychotic about keeping turnover low.  Everywhere I’ve been I try to minimize bureaucracy.   I encourage telecommuting and spend money on great tools.  I like to setup team building, fun activities that appeal to technology workers like playing video games together once a week.  These are all downstream things that I focus on to maintain my upstream asset: people.

What does working on a great team with great people look like?  For students still in school, this is hard to visualize.  Group projects are almost universally hated.  Internships generally involve working as someone’s grunt on some meaningless or semi-solo system or task.  But I got all heads nodding when I asked if while they were doing their horrible group projects with their horrible group members if they remembered that one group who got up in front to present that was clearly happy to be working together, who had enthusiasm for their work product, and had spent an inordinate amount of time on their project.  Did they seem tired or annoyed?  No.  Was their project the best in the class?  Yes.  That is what a well-gelled software team feels like, looks like, and that’s the kind of product it produces. I talked a little more about great teams and how to recruit them and run them as we all munched on pizza.   Afterwards the organizer walked up and stated that I had given the most non-technical tech talk ever, but that he felt it was important and something that often gets missed.  

I definitely agree about the missing piece, and we got invited back for more pizza and another tech talk so we’re grateful for another opportunity.  I’d say that was mission accomplished.

Tivo Premiere Review

After our third busted Comcast provided DVR (every one of them being a different model, with different capacities), we decided to get a Tivo.  I’d long been after one, mostly because I’ve long wanted to be able to actually..uhh..SAVE shows that I might want to watch again without leaving them on the unit.  Ever since NC State’s triumph over UNC got erased from our DVR a couple years ago back, I’ve been a different man.  I need to relive these moments from time to time, usually when my good friend Dan (UNC fan) comes over for any reason.In the past, I’ve been successfully spooked by both Google and Apple and avoided buying a Tivo precisely because I thought either of those outfits would have come out with a solution by now, but neither company has executives that care about sports.  If they did, they’d care about live TV, but they don’t, and instead are focused on delivering TV Shows and other serial type shows over the internet.  This works great if you don’t like sports, don’t watch sports, don’t care about sports. It doesn’t work at all for watching sports, which for me fall into two categories: must-watch-live and can-wait-until-later.  Wolfpack basketball and football: category one.  MotoGP, Soccer, baseball playoffs, Tour de France: category two.  Both categories must have HD, and 720p is not HD.So for my setup, here’s what I need: high definition video capture, with the ability to offload shows in HD to my Network Attached Storage.  I can then transcode and eventually playback any stored media using my very impressive LG Blu-ray player for the TV, or on any other internet connected device.  I also needed the ability to quickly and easily schedule things via the web.  Tivo provides both of these capabilities and does so fairly reasonably.  That’s good.Now for the bad:  the interface mostly sucks.  And the product is completely unfinished – half the menus are in High Definition, and half are in Standard Definition.  For the life of me, I can’t understand why they wouldn’t convert them to be completely HD.  It seems like it would be more complicated to support two menu systems, but there’s probably more than meets the eye.  The box is sluggish too with the menus.  There also seems to be a host of missing features that should be no brainers to support.  Here’s a list, guys at Tivo, in order of preference:

  • Slingbox: there’s an entire company out there providing a set top box to stream content over the internet.  Do this too.  It’s valuable and means one less device I buy.  You can even route the service through Tivo.com and charge me a premium to not munge with my firewall settings and provide a nice Content Delivery Network optimized experience.
  • More one-click operations.  I should be able to delete shows in the list, or select multiple shows and then click delete.
  • The guide seems poorly setup for sports viewers (and trust me, this is the only demographic that will matter in the future).  Sports is the only reason I still have cable, still pay a premium for HD content, still pay a premium for premium channels, need to be able to offload shows from the unit.  Provide me the ability to choose a favorite team for each major sport, then give me an ESPN like ticker across the bottom with any news, scores, upcoming games, etc.  Provide me an in-program HUD that I can bring up that shows stats loaded while the game is in progress.
  • Set up the box so it can stream from a NAS.  Everything can do this now, even the blu-ray player.  Why can’t you?
  • Ship an iphone application that provides all my settings and tivo.com program listings in one spot, and lets me watch my content slingbox style.
  • If I miss a show (power outage, forget to set it up, etc.) and then later say I want to record the season of “Glee” then automatically download the past few episodes for me if they’re available online.
  • Provide an option to auto-skip commercials.  Content providers don’t like this, so make it an option I can turn on myself, and ship it with the default as off.  Partner with them to give me perks if I leave commercials on, like behind the scenes stuff, and still rip out half the ads.

Right now, the Tivo, TV, Blu-ray, Wii, and other devices are wirelessly bridged from the office GigE network, which sucks.  Early this week, I’m getting two powerline ethernet adapters that should route my network via power outlets for a consistent networked throughput of up to 200Mbit, which should be a huge performance boost, and introduce stability to the entire setup.  I’ll update more when we see how that works out.

Home Backup Strategies and Home Media Storage

Sara and I are starting to accumulate a lot of media.  We like to spend time traveling when we can, and I’m learning how to take decent pictures, and these combined mean we have quite a few photos and this seems to be accelerating.  We both listen to a lot of music.  We both enjoy watching movies and TV shows.  I particularly enjoy watching sporting events in HD and Blu-Ray movies (The Wife couldn’t care less).  We also each have a laptop, an iphone and we’ve got an ipod or two from several years kicking around.  I feel like this scenario is pretty typical for couples/families these days, and is only worse if you’ve got a couple of teenagers to multiply the gadgetry.  We also have a nice television a Blu-Ray player in the living room, but often prefer to watch movies or TV Shows in bed upstairs.For those of us who are the home technology guy, this kind of setup can really be a pain for making sure that media is available, that you can access it concurrently (e.g. if NC State is playing but Sara wants to watch a Blu-Ray title), and that it’s all backed up.  Recently I rethought our entire strategy and implemented the following setup, which I’m really happy with.  I’ll provide more updates as I flesh things out more.BackupsWe had been using a TimeCapsule (1TB) but we were having problems running out of space, and I’d had one die after about 10 months.  For our backups, I had the following criterion:

  • Simple and automated – I shouldn’t have to think about backups.
  • Local and remote – backing up to a time capsule or external hard drive is nice, but not when your house burns down or your car is stolen.
  • Cross platform – I run Windows 7, Mac OS X, and Linux (Ubuntu).
  • Available – I don’t want to have to boot up a machine or have one running just to back up.
  • Media specific stores – ideally, I wanted to be able to segment out certain media (music, TV Shows, etc.) to be separate from our machine backups.  This would let me transfer media between machines and also get around space limitations on individual machines.

The solution I settled on had some upfront costs, but as I often tell people – I’d rather go without food for a month then lose all of my data.

  • Synology DS410 Network Attached Storage (NAS) with four 1.5TB drives in RAID 5 giving us about 4TB of storage.
  • APC Smart-UPS 750
  • Repurposed Apple Time Capsule to function as a Wireless Access Point and Gigabit switch.
  • Backblaze cloud based backup.

The Synology device is an amazing product – it runs  Linux, has a very small form factor, and in addition to providing the RAID capabilities, it doesn’t use much power, is quiet, and includes a great web administration interface that adds things like rsync backup to a cloud, UPS support, firewall, different file sharing protocols, print serving, and DynDNS support.  The NAS also works great with Windows7 Backup and can support about 45Mbps across my home class gigabit networking gear.  Buy this unit now!Backblaze protects against our home going up in a blaze and another benefit of offloading backups to the NAS seems to be that our wireless network doesn’t slow down while backing up any more like it used to with the Time Capsule.Media Access / StorageOne of the best things about the Synology device  is that it functions as a DLNA and iTunes media server for music, video and photos.  This means that now we can save rips, ISO images, and MKV files, and all of our music and access it anywhere in the home from our Wireless network.  It Just Worked and within 20 minutes of setup, I was streaming music to my laptop and our LG blu-ray player browses the NAS via DLNA and can play anything (more codecs than XBMC out of the box, including even ISO images).  I bought a Blu-Ray drive for my desktop computer, and now with MakeMKV, I’m making copies of my Blu-Ray discs, and saving them straight to the NAS where they can be ripped and transcoded for easy access via our laptops, iPhones, TV, or any other DLNA capable device.  Finally we’re at the point we’ve been at with music for quite some time where our entire media library can simply be a box of disks sitting in the garage.  Maybe we’ll show them to our children as they try to comprehend what it was like in the old days.To date my only complaint is that it’s a little clunky making a playlist on the iTunes server for the NAS.  It’s possible, just not drag-and-drop from iTunes possible.  Also, I’m playing around with XBMC on an Apple TV that I happened across, but so far it’s been kind of a pain and the codec support hasn’t been great either – some variants of AVI files refuse to play, which is really annoying.  We’ll see how things progress from that angle.