ET Phone Home

ecobee Mothership Won’t Let My Thermostat Phone Home 16


One of my favorite movie lines is from the 2001 movie Bandits, starring Bruce Willis and Billy Bob Thornton. Billy Bob plays a neurotic genius who, at one point in the movie, states “You know the hardest thing about being smart? I always pretty much know what’s going to happen next. There’s no suspense.”

So at the risk of sounding less-than-humble, this is an “I knew what was going to happen next and I told you so” post.

Last month, I wrote a product review for my new ecobee Thermostat. I still love the product, but in that review I expressed my primary concern with the unit: remote access to the thermostat, and remote control of the thermostat, is completely dependent upon ecobee’s servers remaining online. And, as of today, ecobee’s servers have been offline for at least two days.

I first noticed the problem yesterday when I decided to check on the thermostat in my Utah house. ecobee’s main website (http://ecobee.com/) was still online, but when I logged in with my username and password I got this:

On a positive note, that’s an attractive and well-designed failure notice. It’s a bit less attractive in the iPhone’s web browser:

And as for ecobee’s dedicated iPhone app, it just sits there “Authenticating…” for a while until a pop-up tells me “Error connecting to the server. Data could not be retrieved from the server. Please check your network settings.”

LOL. My network settings are fine and dandy, thank you very much! I can access my email, my blog, my Facebook, my Amazon Web Services account, all of my web servers, and the routers in my Utah house, cabin, parents’ and inlaws’ houses. I just can’t access my ecobee. 🙂

Compounding this issue is that it’s the week between Christmas and New Year’s, so I suspect that more than a few of ecobee’s technicians are likely on vacation. I’ll be keeping an eye on their remote access servers to see how long it takes to get them back online and I’ll post an update here. Maybe we could get a pool going in the comments! 🙂

The “good” news is that I already know when my next visit to Utah is, so I had already placed my ecobee thermostat in Vacation Mode so that it would return to normal programming the day before I got there (I like the furnace to kick in a full day in advance so that hard surfaces such as counter-tops and wood floors have a chance to warm up to air temperature). But if my trip happened to be tomorrow instead, and had I been planning on simply remotely connecting to my ecobee the day before my flight and manually adjusting the settings, I’d be arriving to a cold house… which is precisely what a WiFi-enabled thermostat like the ecobee is designed to prevent.

Please, please, please, ecobee! Please listen to your customers. Or maybe just listen to this one. I just happen to have a wee bit of experience with home automation, remote access, networking, web servers, hosted applications, and technology in general. Don’t get me wrong – I really like your product. It’s a good product, and because you were smart enough to build it with hardware that allows its embedded software to be remotely upgraded and improved, it has the potential to be a great product. I also really like the well-designed and powerful interface of your remote access website. However, as I stated in my earlier review, the power of your product all hinges on a single weak point: the uptime of your servers. And as of right now, they’ve been offline for at least two days that I know of (I don’t know if they were offline earlier when I wasn’t checking my thermostat).

I still want to be able to use your hosted web interface, but I also need to be able to connect to my thermostat directly in situations where your server crashes, or your network connection goes down, or your DNS records get messed up, or a tornado takes out your building, or Jimmy in tech support accidentally trips over a power cord and takes your hosted application offline. Unless you’re willing to invest in the type of infrastructure that can ensure 99.99999999% uptime to your remote access servers, please give us direct access to our thermostats.

I don’t mean to be overly harsh, ecobee. I like you guys. And I wish I’d been wrong. But, at the risk of sounding less-than-humble again, that just doesn’t happen very often. 🙂 Server downtime is a reality that all true geeks must accept. Therefore, taking whatever steps are necessary to ensure that customers can access a device whose primary purpose is to be remotely accessible, even if something goes wrong on your end, should be what keeps everyone at your company up at night – from the CEO to the marketing managers to the engineers to the web designers. Heck, I’d get the janitor on board with the concept, too.

Please allow me to quote my earlier review:

…remote access methods rely on ecobee’s private servers to provide remote access to your thermostat. If ecobee ever disappears, or their web server dies, so does your ability to use this thermostat for its intended purpose. Any device this smart that sits on your network and has an IP address should have a user-accessible web interface. But the ecobee doesn’t, which is a big drawback for me. Just as I can with my router, modem, wireless printer, and a host of other network-enabled devices in my house (including my Yamaha piano), I should be able to access and control my ecobee thermostat directly from any web browser on the local network, as well as set up a port forwarding rule on my router to access my ecobee from any remote browser using a dynamic DNS service.

I stand by those words – and I know you guys want to stand by your product. You’ve got a smart thermostat. It should come as no surprise that smart customers buy it. So if you have smart customers, the smart thing to do is listen to them.

————————

12/30 @ 4:30PM PST: Now when I attempt to login, I’m redirected to this page, which says:

Our Webportal is currently under maintenance. We are sorry for the inconvenience.

That’s different than the previous error message, so I’ll cross my fingers and hope that means they’re making progress on a solution!

12/30 @ 7:19PM PST: I’ve received two emails from ecobee so far today thanking me for this blog post and assuring me their development team is feverishly working on a fix. I’ve been in their shoes in the past, and I know it’s not fun.

12/30 @ 9:37PM PST: Ecobee’s MAIN site (http://ecobee.com/), which didn’t seem to have any issues before now, now looks hosed when viewed in my primary browser, Google Chrome. I checked and it looks fine in Firefox and IE (even after clearing my cache), so either Chrome is connecting to an out-of-date version of the site on a cloud-based edge server, or their server is sniffing for browser strings and feeding the wrong info to Chrome (that’s my guess). Chrome is seeing a version of their site where all the URLs (including links to CSS) are the wrong local hostname (just “web2”) with no TLD string. I’m assuming web2 is an intranet or dev site hostname, which probably works inside ecobee’s network, but breaks for the rest of us. Their stylesheet, for example, is trying to pull from https://web2/wp-content/themes/ecobee/style.css. Bummer. But fortunately, that’s an easy fix (and I just emailed them to alert them).

The maintenance message after attempting to login to the remote management interface is prettier now, and says:

We are currently experiencing issues with the ecobee servers and our Development team is working on the resolution. We apologize for this inconvenience and if you would like to be sent updates on our progress please email [email protected] and we will add you to our notification list. Thank you for your understanding. The ecobee Team.

I emailed and asked to be put on the notify list.

12/30 @ 9:43PM PST: ecobee just sent this announcement to the customers on their notify list:

ecobee Server Update – December 31, 2010, 1:00 am EST

Thank you to all of our customers who have shown patience through this period, we appreciate your understanding and support. The Development team has been working on the issue and has isolated the problem which is great news. We are now bringing a select group of thermostats online as part of our due diligence and we will send another email informing you of when this process is complete. In response to some concerns that have been raised, your ecobee Smart Thermostat will continue to run the program you have created during this period of limited web accessibility.

12/31 @ 6:09AM PST: ecobee just sent this announcement to the customers on their notify list:

ecobee Server Update – December 31st, 2010, 9:15 am.

Good Morning;

The Development is continuing to make progress on bringing a limited number of test units online but we have are not able to reinstate full web portal accessibility for all customers at this time. The entire team is continuing to work on the issue and will do so until there is a resolution.

Thank you to those who have offered their support and encouragement and we are sorry for the inconvenience that this is causing to our customers.

12/31 @ 9:45AM PST: ecobee’s main website works in Google Chrome again. The maintenance message on their website has been updated to say:

We are currently experiencing issues with the ecobee servers but we have isolated the issue and the Development Team is running their final due diligence. We expect to have all of our customer’s web portal access available in the next 2 hours.

they also just sent this announcement to the customers on their notify list:

ecobee Server Update – December 31, 12:45 pm EST

Good afternoon,

We have isolated the issue and the Development Team is running their final due diligence. We expect to have all of our customer’s web portal access available in the next 2 hours. Thank you for your patience during this period.

Happy New Year.

That sounds encouraging! However, I can’t help but think that during a technical outage (of which my companies have had their fair share), I would never include a specific deadline on a website or email message (such as “in the next 2 hours”), no matter how certain I was that we would hit it. Optimism is great, but I’ve said stuff like that in the past, and then ended up with egg on my face 3 hours later. Until everything is actually up and running again, I prefer to stick with “as soon as possible.”

12/31 @ 11:52AM PST: ecobee just sent this announcement to the customers on their notify list:

ecobee Server Update – December 31, 3:00pm EST

The issue we have been experiencing with the ecobee server has now been resolved. All customers should have access to their web portals. Please note that during this interruption some of the reports data may not have been captured resulting in the reports graph having no data at certain times.

Thank you for your patience during this period and a Happy New Year.

Hooray! I have verified that I can login and access my thermostat and reports. They were less than 15 mins late on their 2 hour promise, so that’s commendable. Surprisingly, I’m only missing reporting data from around 2:45AM to 6:30AM yesterday morning. All of my programming data and other settings appear to be intact, and everything appears to be back to normal. I really didn’t want to have to write a tongue-in-cheek blog post entitled “My ecobee has been offline since last year!”

12/31 @ 2:06PM PST: ecobee Founder and CEO Stuart Lombard (who had commented earlier on this post… see below) sent the following open letter via email to all ecobee customers:

An Open Letter to ecobee Customers:

I am a founder and President of ecobee. Please accept my sincere apologies for the issues we have experienced over the last two days with our web servers. I believe that, up until Wednesday, we have had an excellent uptime record (over 99.9%). Unfortunately, on Wednesday we ran into a performance-related issue and the resolution was not straightforward or easy to isolate.

We had a dedicated team working around the clock to resolve the issues, and we are now operational again. We will review our processes thoroughly to determine what went wrong and how we can prevent this situation from happening in the future.

We are extremely upset with the downtime. We are committed to delivering excellent uptime and reliability, and we expect that to continue in the future. We appreciate your patience and ongoing support.

If you have any questions or comments, please do not hesitate to contact me directly.

Best regards,
Stuart

A well-written letter saying exactly what it should for the average customer. Of course, über-geek customers want to know the gory details of the outage, but an open letter isn’t the right place for that. The comments section of an über-geek’s blog is the right place for that (hint, hint).

Good job getting it sorted out, guys! Look on the bright side: tomorrow, you can say you’ve had 100% uptime in 2011! 🙂

Go enjoy the New Year’s holiday. You’ve earned it. But when you get back in the office, I’m still dying to know exactly what went wrong, and what you’re planning on doing (in geek talk, not marketing talk) to decrease the chances of it happening again.

And, yes, I still wish we had direct access to our units. 🙂

10/23/2011 Update:

Due to repeated requests about an Ecobee users discussion forum, I’ve started one myself. If you’re an Ecobee owner, enthusiast, or would just like to join the conversation, you can subscribe here:

Google Groups
Subscribe to Ecobee Users
Email:
Visit this group
  • Pingback: Product Review: ecobee Smart Thermostat | Steve Jenkins' Blog()

  • Chris Carradine

    Steve;

    Hi Steve,

    This is Chris Carradine from ecobee, you are correct are servers are down and even though it is the holidays our development team is working on it as I type. We sent an email to everyone who contacted us earlier this afternoon letting them know we are working on the issue and we will continue to update our customers on our progress.

    Appreciate your patience.

  • I am a founder and President of ecobee. Please accept my sincere apologies for the issues we are having with our web servers. We have a dedicated team working to resolve the issues as soon as we can – we are extremely upset with the downtime and our goal is to be fully operational as soon as possible.

    I believe that, up until yesterday, we have had an excellent uptime record. Unfortunately, yesterday we ran into an issue and the resolution has not been straightforward. We are committed to delivering excellent uptime and reliability and we expect that to continue in the future.

    With respect to your comments regarding listening to our customers, we believe we do. Many of the features you see in the product have come from customer suggestions and our goal is to get as much feedback as possible. With respect to your specific feature requests we will certainly consider them as part of our product roadmap and we appreciate those suggestions.

    Best regards,

    Stuart

    Stuart Lombard [email protected]
    President and CEO
    ecobee | green made easy
    333 Adelaide St. West | 6th Floor | Toronto | ON | M5V 1R5
    T 416.987.1052 C 416.209.9304 F 866.592.7344

  • Hi Chris, great to hear from you again, and I’m impressed that you responded to my email request to be put on your notification list at 11:30PM your time this evening. That’s awesome. That makes you one of the 1%.

    Hi, Stuart. It’s a pleasure to “meet” you. 🙂 As a serial entrepreneur, I always admire others who are crazy bold enough to turn an idea into reality. I’m a big fan of your maiden product. Congratulations on your success so far.

    I appreciate that both of you took the time to write comments to this post. Happy New Year to you both.

    I’m sure it’s a bummer for your dev team to be working unexpectedly through a holiday, but I’m certain I speak for all of your customers when I say that we appreciate the fact that they are. I tend to take a bit heat from my wife when I sneak away to my computer on Thanksgiving, and Christmas Eve, and Christmas Day, and.. and… 🙂 But the Internet never “closes,” so therefore some of us never get to truly clock out. That’s part of my 1% rule (for any visitors reading this who don’t know why I keep saying “1%,” use the built-in search feature of my blog and search for “BYU Lecture”).

    I appreciate the apology, Stuart – but none is needed. As I’ve said before, technology breaks. Of course, initial quality is important, but the mark of a truly good company isn’t whether they can make products or services without any flaws, but rather how their people are prepared, positioned, and empowered to respond when flaws inevitably arise. Every hard drive crash, accidentally deleted file, system failure, fried motherboard, and yes – even a power cord tripped over by a tech support staffer (I wish I were making that one up) at every one of the companies I’ve managed has made us better prepared for the next time the inevitable occurs. Of course, right now you just want to get it fixed. But I’m confident that when the dust settles you’ll have unique perspective on how to prevent having to fix this problem again in the future, and probably prevent a few others, as well. May I suggest an embedded interface on the device itself with direct remote access? Oh… you’ve heard that one before, eh? 😉

    I’m glad to hear that customer suggestions help drive your development path. I haven’t been a customer of yours long enough to have experienced a software update myself yet on your iPhone app, Web interface, or thermostat firmware (or if I have, you made it smooth enough to not be noticed), but I do look forward to that. I believe loud-mouthed and annoying customers (like me) are generally more valuable in the long-term as idea sources than the quiet ones (although sitting in the CEO seat, sometimes I’ve really wished for just a few more of the quiet ones). I appreciate the tact with which you’ve handled my terrorist demands polite suggestions thus far. I’ll admit that I’m quick to point out when things don’t go exactly how I want them to in a business setting, but I’m just as quick to give credit where it’s due. My kids would tell you it’s a lot like how I deal with them, too. 🙂 The fact that the two of you responded (as did Heather via email), and that Chris is replying to emails at 11:30PM deserves credit.

    From the standpoint of a technology sadist, I have to admit I’m dying of curiosity to know exactly what went wrong… but I probably love a technology train wreck more than most. If you’d rather not disclose that, I’ll understand. And while I’m sure you’ve got things under control, and likely have the fix in the works, I’d also be willing to sign an NDA and be another set of eyes on the problem, if I could be of any help. I’ve been at the helm of a few of those wrecked trains myself.

    May the Force be with you and your dev team in this dark hour. 🙂

  • Pingback: Installing my 2nd ecobee and Using Group Admin Features | Steve Jenkins' Blog()

  • Jim

    Steve,

    Thanks for the very informative blog on the ecobee. I had researched this thermostat thoroughly and had decided against getting it, specifically due the the reliance on ecobee’s servers for remote access. Your recent experience is exactly what I had feared would happen. It looks like a very cool product, but at that price point to be effectively “locked out”, remains a stumbling block.

  • Mario

    Steve, wonderful posts. Thanks for taking the time.
    If you had to do it all over again, would you stick with the Ecobee? Or was there a better choice given the reliance on their servers.

    • I would, Mario. Their interface is light years ahead of the competition, and they seem very driven to be the market leader. I also like their approach of using a separate interface module and thermostat unit, with all the wiring going to the interface module and all the processing and communications done on the thermostat. I like the fact that they are running a real embedded operating system inside the thermostat, meaning firmware updates are easily pushed out to their customers. This means that a) they could eventually add a self-contained embedded web interface so that the unit doesn’t depend on a central server (and with enough customer demand I think they’ll eventually offer that), and b) someone will eventually figure out a way to hack into the device and get a shell prompt, opening the door to 3rd party developers to create their own firmware and interfaces for the unit, much like the open source firmwares for the popular Linksys WRT54G series (Tomato, DD-WRT, OpenWRT, etc.). The reason the aftermarket router firmware market exploded is because the routers became very popular but customers wanted more functionality from the unit than Linksys provided. The ecobee does a lot, but it has the necessary hardware (processor, memory, WiFi antenna) to do even more, so if ecobee doesn’t developer the features the customers want, someone will eventually figure out a way to take advantage of their powerful hardware platform to hack their way around it (I know a couple people who are working on it as we speak, in fact). The same thing happens with any “connected” hardware device that reaches a wide audience: iPhone jailbreaking, cell phone carrier unlocks, TiVo hacks, router firmware hacking, etc. Linksys was smart enough to release a version of their flagship wireless router (the WRT54GL) that was designed especially for those aftermarket firmwares, because they make their money from selling hardware. If the ecobee gets enough market penetration, I don’t see how they’ll be able to avoid the same thing happening to their devices.

  • Ben

    Steve, It is a matter of time before these devices get hacked. I’ve been a customer for years and Ecobee’s software is great, but evolves rather slowly. Once the customer base gets large enough, people will be frustrated with the frustrating aspects of the unit and hack the unit to open it up. Take the new RSM unit the was released for the Ecobee unit. The Remote Sensor Module is capable of taking in input from CO2 sensors but this functionality is disabled in the Ecobee and limited to their commercial version the EMS. The Ecobee could be a great spring board to home automation, but the software is holding them back. I think that this is done to stick with the “green made easy” mantra. Dumbing down the UI and limiting functionality to keep things simple is frustrating to advanced users. Let’s be honest… People who spend this much on a thermostat are typically more advanced users who are looking for more than your off the shelve thermostat. Added functionality would blow these peoples minds.

  • Mike

    Steve,
    Thanks for the thorough review of the Ecobee T-stat. I am considering buying several of these for my Church. Have you had any more experiences (good or bad) with yours? Have you heard if Ecobee is going to use your suggestion of making these units browser accessible, without using Ecobees website for access?

    Thanks again,
    Mike

  • brizone

    Just bought this system, the iPhone app hasn’t worked since it was installed nearly a month ago. Very annoying. Seems like a very low-quality touch screen too, very difficult to get their bizarre scrolling system to work or for buttons to toggle, etc.

    I’m very concerned that this company may not survive for much longer, and then I won’t be able to get at my *expensive-as-hell* thermostats wirelessly at all. It’s mind-boggling that a company would force you through their website in 2011 for an IP-enabled device that could easily just serve things up directly.

    Having serious second thoughts, I may have them rip these out and refund me, and just wait for another company to come along in the next year or two that’s ready to implement some more common sense open-platform approaches. Bummer.

  • Pingback: Installing Ecobee Smart Thermostat #5 | Steve Jenkins' Blog()

  • Jenny Lomar

    You should have bought a Nest.

    • Nah. After comparing the two, the Ecobee is still miles ahead.

  • Chris Authier

    We are a mid sized manufacturing company and it was decided to go with ecobee thermostats for our building hvac.
    We have 4 of them setup now and just discovered that we cannot set the hostname on the thermostat. To me that seems like a pretty basic network setting. At the very least I would expect that it might be populated with the thermostat name.
    We hope to have 8-10 of these when we are finished, using DHCP. Tracing a thermostat through logs using a dynamic address certainly adds time to troubleshooting.
    What do you think?

    • Hi, Chris. I run three ecobees in my house and solved this by setting up DHCP reservations for each of them, then setting local hostnames for them in the local DNS server. Now I know exactly what IP or hostname to ping to look for in logs! 🙂