Month by both uptime report

Observu has long enabled you to view uptime over past 24 hours, 30 days and year. As well as view day-by-day uptime over the past week, month, quarter or year. However we were missing a month-by-month report of uptime to see more general trends.

We’ve listened to our client feedback and added a new monthly summary report to the ‘Uptime Reports’ page of your monitors.

Monthly uptime summary

We are also working to add more advanced uptime statistics to other reports. If you have any suggestions on stats you would like to see, please let us know.

Posted in Progress Report | Comments Off on Month by both uptime report

Introducing the Observu App

Last week we release the first version of the Observu App. The app lists current and most recent incidents for a quick status check. If there is anything actionable, those items will be prioritised on the home screen.

Monitoring Alerts via Push Notification

Push Notifications

The main feature to install the app for is the ability to receive push notifications. These can be set up in the same way as our other notifications, with configurable thresholds and intervals.


The app has been implemented in C# using Xamarin.Forms, allowing for quick cross-platform development. Push notifications are powered by Amazon SNS.

Get it now

You can download and install Observu for iPhone and iPadObservu for Android


We plan to quickly extend the app to also include detailed reporting regarding current status and historic data. We welcome all feedback.

Posted in Howto | Comments Off on Introducing the Observu App

5 reasons your developers need to do first-line support

As a developer, I don’t particularly enjoy customer support. Questions are either about things that are obvious (or should have been) or about things that I don’t have a clue about either and need a lot of investigation. Some customers are just plain rude with “it’s not working” in the subject and nothing else.

At first sight it seems to be a huge waste of your expensive developer time to spend time on this. However, I strongly believe you should. (And this holds for everyone involved in the development of your product, not just the ones that write code)

1. Developers can ask the right questions

Users don’t always know how to ask the right question. A developer that actually understands the technology and implementation of your product is much more able to aks the right questions for clarification. In this way you prevent a lot of unnecessary back and forth of irrelevant questions before escalating the problem. No matter how well you’ve trained your support staff, a customer can always tell if you asking standard questions.

Developer answering customer support tickets

2. Developers can provide the right answers

There is no-one better at actually telling you how you supposed to use a system than the person that built it. Even if you can’t solve the issue immediately, telling a user what actually went wrong will make them feel like their input is valued and taken seriously. Furthermore, many issues can be resolved on the spot or worked around, instead of having a ‘escalated to the tech department’ and waiting for a week for a follow-up. Users will notice a personal response with actual substance and will turn them into your most loyal customers.

3. Developers will learn about your customers

As a developer, it’s easy to get lost in just crunching on lists and lists of features and bugs. Product development is not a vacuum. There is no better feeling than actually solving a problem for a customer and being able to answer them that the issue is already fixed. Being in touch with customers will help fuel ideas about new features to make life easier for everyone involved.

4. Developers can stop the questions

The most powerful action a developer can take is to actually prevent the question from re-occuring. Of course bug reports will get to the developers independent of whether they answer the reports directly. However, there is more to support than bug reporting. The standard way support will deal with re-occuring questions is to provide canned responses. Instead of that a developer should be able to take away the root cause of the question even if it is just by providing some extra help information in a form.

5. Support requests help set priorities

One of the hardest problems for developers is to get a feel for priorities if your backlog grows faster than you are ever able to cross items off. Of course you need someone managing those priorities from a business perspective, but there is a difference between your product owner telling you something is important and to actually feel it is important. Interacting with customers can greatly improve this internal compass.

Disclaimer: I’m not talking about phone support, that might be too much of a disruption. However checking and answering support tickets a few times a day can be worked into anyones schedule.

Posted in Opinion | Comments Off on 5 reasons your developers need to do first-line support

Monitoring private websites and keeping credentials secure

Today I encountered this question on stackoverflow. It asks about monitoring tools to run locally because they cannot share credentials with third parties. (like Observu) This comes with the disadvantage of now having to take care of the reliablity of this script, making sure you receive alerts if anything goes wrong and storing historical data for later root cause analysis.

As this is a problem that is very relevant to our users as well, I will expand on the problem a bit more. First, if possible, the best approach would be to provide monitoring with a dummy user that has no sensitive data in their account. This is often possible if you are running an account based service, where data is owned by the user that is logged in. If that is not possible, I can think of a few solutions to keep your passwords or API credentials to yourself, while still using a remote monitoring service:

  • Create a separate page on your website, that executes all critical operations, but just does not provide any relevant content as a result. e.g. just have it output a keyword and a status for each operation if it succeeded. You can then have a remote monitoring service call that page and use regular expressions to parse out those keywords and collect the status on each one of them as individual data.
  • Similar would be to use the language that you are building your website in to call it’s own URL’s and do the regex parts and again build a status overview. This however does come with the downside that the URL’s are called locally, so firewall rules, etc. may be different than for a real user

The other way would be to run just the part that calls your website on your own VPS (or preferably multiple ) that you can secure the way you like. And send the results to the API of a monitoring service like ours ( The script that does the actual fetching, querying and testing it with regular expressions can be in any language you are comfortable with. (Or you can just take the results of calling JMeter)

The advantage here would be that you call your scripts from multiple remote locations, while still keeping control of the passwords. (Given that you either have your own datacenters remotely or trust at least Amazon or any other provider enough to have your passwords in a small virtual instance)

If you are interested in this approach, we are very willing to provide you with assistance in setting this up. Just send an e-mail to our support department outlining your case and we will point you in the right direction, providing ready-made scripts when possible.

Posted in Howto | Comments Off on Monitoring private websites and keeping credentials secure

How to resolve ‘Got error 28 from storage engine’

A common cause of website and webservice failure is running out of disk space. Either the webserver is no longer able to write to it’s log files and fails completely (resulting in an error page or no connection at all) or the database server may return an error. For example MySQL returns the fairly cryptic: Got error 28 from storage engine.  If you are using availability monitoring, it will start alerting you because your page is no longer showing up properly.

Basically meaning the database server can’t write either new data or temporary files (often needed for complex queries). Because the error does not happen on all SQL queries, the error may only become visible on certain pages or actions.

Worst case, the inability to write data, may lead to database corruption, requiring a repair after you freed some space. This may be a problem if you have large MySQL MyISAM tables, because repairing those requires additional free space as much as the largest table you’ve got. The one thing you just ran out of, causing the automatic repair to fail. How to repair MySQL tables will be subject of another post.

After you became aware of the above issue, it’s important to find out what partition is full and what is taking up all that space. I’m assuming that you are on a Linux-based server.

The first thing is to find out the partition in trouble, by running:
df -h

This should show you something like: (note the 100% on rootfs)

Filesystem Size Used Avail Use% Mounted on
rootfs 20G 20G 0M 100% /
none 6.3G 344K 6.3G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 32G 176K 32G 1% /run/shm
none 100M 0 100M 0% /run/user
/dev/md2 92G 829M 87G 1% /data
/dev/md3 1.8T 155G 1.5T 10% /home

To figure out what particular files are big, you could run: du -hs * in root ( / ) and then descend into the biggest directory and refine. However, on a production server, this will often take too long and should only be used as a last resort. Most often running these three should already tell you what is going on:

du -hs /tmp => if this is particularly big, you are probably not properly cleaning temporary files after use, or writing logs that are never rotated
du -hs /var/log => if this is particularly big, you may be keeping log files forever or some log has gone haywire, you could consider transporting logs to an external server for long-time archival
du -hs /var/lib/mysql => this is your mysql database, usually you can’t do much about this, except move it to a different server or partition
du -hs /var/* => if none of the above, often it’s still somewhere in this part of your file system

What often happens is that the server comes with a certain partitioning and disks by default, that has a small root partition and sometimes also a small partition just for /tmp. Default installs usually put their database files onto the root partition, so if your database is particularly big, you may quickly fill up the root partition with database and log files, while there is sufficient space somewhere else. This means you will need to move your database and then use a symlink (ln -s), mount –bind or change your server configuration (edit the mysql.conf file) to point to the new database location.

Running out of space happens to everyone at some point, however to avoid a lot of stress, you should make sure you know in advance. This will prevent errors on your service and potential data corruption. To do so, you can use Observu server monitoring, which will set alerts at specific levels of disk usage. The notification will include information which partition is almost full and needs your attention. To resolve the issue you can then (without stress) use the hints provided above.

A special recommendation for those still using MySQL MyISAM tables: make sure you leave enough space for repair/recovery. This means you will need to set warning levels as early as 60 or 70% of disk usage if table size is unbalanced.

Posted in Howto | Tagged | Comments Off on How to resolve ‘Got error 28 from storage engine’