UPDATE: 0800 pacific time, Nov 12
All Zoho services are now functioning normally. We have performed many upgrades and are monitoring carefully. We’ll make a detailed statement soon.
UPDATE: 1605 pacific time, Nov 10
Finally we have made progress with Mail! Mail should be clearing now and reaching you. The protections we put in place to mitigate DDoS attacks had the side-effect of affecting mail delivery. We have identified these and cleared them so things should be working now.
Please tweet us @zoho, or call support, if you’re still not receiving mail. Your business runs on Mail and we can completely understand how stuck you must have felt. Thank you immensely for your patience and understanding.
UPDATE: 0915 pacific time, Nov 10
We continue to be on very high alert, watching all services very closely. The last several hours have been focused on performance issues and bottlenecks that many of you customers have reported over the last several hours.
UPDATE: 0615 pacific time, Nov 10
Due to the emergency rerouting – designed to mitigate the attacks – some links still have issues. As a result, you will see some slowness and delays on mail delivery. We’re working on this and will keep you posted.
UPDATE: 1815 pacific time, Nov 9
The attacks continue and we are managing to stay afloat. We are not assuming that the worst is over yet and customers should not either. Stay with us.
UPDATE: 1545 pacific time, Nov 9
Almost all services have been up all day, today. Services are all much slower than usual, although we can see some improvement over the last few hours. Incoming mail is extremely slow. Many customers have pointed this out and we’re deeply sorry that it’s directly affecting your businesses. Our team is working specifically on this issue right now.
Please read the other post in this blog, Answers to frequent questions, to get answers to specific issues. We will continue to update that as well.
UPDATE: 1034 pacific time, Nov 9
Most services are up, not all. Service continues to be slow. Working on it.
UPDATE: 0847 pacific time, Nov 9
Here is what is happening with our infrastructure. We had several pending updates that were designed to mitigate DDoS attacks. Those updates were scheduled to go live in the next 2-3 weeks. We have had to squeeze them in over the last weekend. That has created a bit of instability in our systems, given the emergency nature of the upgrades themselves.
So while we have taken successful counter-measures against DDoS, we are now working on strengthening our systems. We have traffic rerouted through our secondary data center and that introduces more hops that adds latency (delay) as well. We are working on all these issues right now. Thank you for your patience.
UPDATE: 0650 pacific time, Nov 9
Due to relentless attacks we have taken emergency counter measures. Part of this is to reroute traffic and data, through additional network hops to filter out the attack. This added complexity is making service access unstable and slow for customers. We are working on it as we speak.
UPDATE: 2015 pacific time, Nov 8
Services are up and we are mitigating the attack. We fully expect them to keep coming though and disruptions may yet occur.
UPDATE: 1705 pacific time, Nov 8
Zoho services were targeted again on November 8 at 1705 hours pacific time (0115 GMT). Services were down for about 33 minutes and came back up. Many are not reliably up yet. Our teams have been working nonstop over the last 72 hours with various counter measures. We are still at it and are prepared for tough days ahead. We expect more attacks and more service disruption as we get into the work week. Please stand with us at this time.
Zoho under distributed denial-of-service-attack: Nov 6
Dear Customers and Well-wishers,
Zoho was subject to a criminal attack to our networks called a distributed denial-of-service attack (DDoS). This started at 8:15am pacific time on November 4, 2015 and has continued intermittently. The obvious intent is to make Zoho and all of our services unavailable to customers by flooding our servers with bogus requests from multiple sites. The attack was accompanied by threats and a blackmail attempt at extortion to prevent ongoing attacks.
This attack is focused on denying access since it targets the network connections to our servers. All your data are sound and secure, but unfortunately you cannot access it reliably. This is like a crowd of people standing around the entrance to your bank, preventing your entry. Your money is in there and safe, you just can’t get at it until we have them all moved.
Companies like us, that have offered services accessed through the internet for many years, carefully prepare for and expect these attacks. But the attacks are getting worse and more sophisticated. In fact, there have been other attacks this very week. Secure email providers like ProtonMail and Runbox have been hit. Many major banks and businesses with online customers have seen these attacks in the last year.
We’re working round the clock, with service providers, experts, and others to continuously improve our defenses. We are also contacting law enforcement. There is no single silver bullet to fix these issues and we have to work at it diligently and with the right experts behind us. We expect that these attacks will continue, but we also expect to prevail.
In the meantime, things will be rough for you, and you have our deepest apologies. The most unfortunate part is that we cannot tell you exactly when everything will be back to normal. We simply cannot know. We regretfully ask for your patience in advance, knowing fully well that you rely on us to run your business and to serve your own customers. We are deeply sorry for your trouble.
Please stand by us as we fight this attack. We cannot give in to criminals and embolden them to perpetuate other attacks. Thank you again for placing your trust in us.