A Programmer’s Survival Guide to Calendaring

Getting Started

You decided to build an application that reads, and maybe writes, calendar events. Congratulations! There are a few things you should know.

There are multiple calendar providers. The main players in the enterprise world today are Google and Microsoft. Google offers a single service, Google Calendar, with a single JSON/HTTP API. Microsoft offers also offers a cloud-based calendar service similar to Google Calendar, branded Outlook.com for free users, Office365 for paying users as well as some other names. They are accessible via a JSON/HTTP API. However Microsoft also offers on-premise email hosting, branded as Microsoft Exchange Server, to not be confused with Exchange Online which is yet another brand for Office365. These self-hosted mail servers are still common as of 2017 so there’s a good chance you’ll have to support those users. The API to talk to these servers is called ActiveSync (EAS) and is difficult to deal with. Other large calendar providers include iCloud Calendar and Yahoo Calendar, both supporting the CalDAV standard.

If you want to avoid having to deal with all of these different APIs (and we recommend avoiding ActiveSync especially), consider using a service that exposes a unified interface. We currently use Nylas. It also provides access to email. Another service that we haven’t tried is Cronofy.

None of these APIs is super fast. An API call takes around 1 second, so your client should minimize the number of sequential requests to calendar APIs in context where latency matters. The best ways to make requests faster are:

  • Have the client load and display event lists incrementally e.g., fetch and render a list of events “page by page”, each of these calls to your server corresponding to just one call to the upstream calendar server.
  • Run requests in parallel when possible e.g., when fetching lists of events from multiple calendars before merging them.

As a last resort, you may have to cache some event data. This is hard to do correctly because of synchronization problems. Many calendar providers offer an API (like Nylas’s delta API) that calls a webhook on your server when calendar data changes. We don’t recommend starting off with webhooks for most calendar apps though. Webhooks don’t always fire when you expect them to, and if your server misses or errors on any webhook calls, your local cache may be inaccurate. Our experience that retrieving a month’s worth of calendar events for most providers rarely takes more than a few seconds. So unless your application really needs to frequently read event data for a very long period of time, you might want to avoid webhooks at first.

Typical Calendar Structure

It is safe to assume that all the leading calendar providers expose calendar data as follows:

  • A user can be identified by their email address.
  • Each user has access to a set of calendars.
  • Each calendar has an owner.
  • Each calendar has a globally-unique identifier and a title.
  • A calendar holds a set of events.
  • An event has an owner, who is not tied to the owner of the calendar.
  • Each event has a globally-unique identifier.
  • Each event has a start time and an end time.

Universal Operations Supported by Calendar APIs

  • List calendars accessible by the user.
  • List events that are contained or overlap a given time window.
  • Fetch a single event from its ID.
  • An option to expand recurring events into individual instances or to only return the master of the series.

Various Gotchas

  • Permissions and visibility are managed differently from one platform to another.
  • Not every user sees the same data when requesting the same event.
  • Free-busy access is a read-only mode for accessing a calendar where all details except the start and end time of the events are hidden from the requesting user.
  • Different users may see different data for the same event. In Google Calendar for example, if user A shares an event with user B, then B can edit the description field for themselves but A will keep seeing the original version.
  • Bugs exist, such as malformed timezones or recurrence rules. Can you spot the error in the following recurrence rule?
    RRULE:FREQ=WEEKLY;UNTIL=20171208T235999Z;BYDAY=WE,FR
  • HTML event descriptions are used by some providers (Microsoft), making them hard or impossible to edit programmatically because it’s not clear what’s legal to edit. We haven’t managed to edit HTML event descriptions successfully with Nylas.
  • Calendar applications typically create events with an empty guest list. The calendar owner is added to the guest list automatically only after another guest is specified, resulting in two guests. It makes it hard or impossible to determine who’s the person involved in the event because (1) the calendar owner is not known unless the accessor is also the owner and (2) the event creator is available but they may be an assistant rather than the calendar owner.
  • The guest list of an event needs to be cleaned up. In Google Calendar, some items are not guests but so-called resources (such as meeting rooms) that should be filtered out.

Timestamps and Timezones

Timestamps

All of your machine-readable timestamps represented in text-based formats (JSON, logs, …) should use the RFC 3339 date-time format e.g., 2017-07-25T13:16:42.792-07:00. Such a timestamp represents solely an instant in time. Your application should treat 2017-07-25T20:16:42.792+00:00 and 2017-07-25T13:16:42.792-07:00 interchangeably (with the exception of guessing a time offset, see below). JavaScript:

> new Date().toISOString();
"2017-07-25T21:54:30.611Z"

Note that some applications out there show the number of seconds since January 1, 1970 UTC with simple decimal formatting; this gives us 1501013802 for the previous example. As you can see, a human can’t make sense of such date without tools, which is problematic when inspecting data and logs that are meant to be human-readable.

Timezones

In the context of calendaring, you must deal with timezones because people travel or have calls with guests far away. Luckily, it’s easy to do correctly.

First, you need to know the difference between a timezone and a time offset. A timezone is a region of the world associated with rules that allow deriving an offset with respect to universal time (UTC). Timezones have unique identifiers such as America/Los_Angeles and can also be designated by aliases such as US/Pacific. The former is the recommended form and has the structure Continent/Largest_City_In_Timezone. Three-letter codes such as PDT are ambiguous and should not be used in machine-readable representations of timezones.

Time Formatting

Formatting time should be done with the time offset that matches the location of the context. An offset can be obtained from a timestamp and a timezone. It cannot be obtained from a timezone alone because of possible Daylight Savings rules or because rules may change over time.

In any case, it should be made clear to the user which timezone is used.

  • For viewing the day’s agenda, the current location of the user determines the best time zone.
  • For scheduling future events in a remote location, the local time at that location is most useful.
  • For flights across timezones, it’s best to show the start time and the end time in their respective timezones, as well as the trip’s duration.
  • For a single in-person meeting, the time should be formatted according to local time. If the user lives in Los Angeles but attends a meeting in New York City, local time is New York’s time.
  • For a call involving guests in multiple timezones, use the organizer’s timezone. In this specific case, it may be useful to also display times in the user’s timezone at the time of the call.

While this is kind of nice in theory, in practice we often don’t have all the data needed to follow the suggestions above.

Obtaining an Appropriate Timezone

Your application that involves calendars should ask the user for their timezone during the setup phase. Timezones may be available from the calendar API, but they are associated with each calendar rather than with the user, so if a user has access to multiple calendars, this doesn’t tell us which of the timezones is correct. It’s a good idea to guess the timezone using tricks that may be available, but it requires user confirmation.

A timezone can be obtained from a location using a service such as the Google’s Timezone API. Here are ways to guess a user’s timezone:

  • From the user’s geographical coordinates, if available from the environment.
  • From the client’s IP address, if available; then use a service to resolve it into geographical coordinates and a timezone.
  • From the time offset used by the web browser to format dates. This solution involves computing the difference between UTC and local time. It’s taken care of by JavaScript libraries such as Moment.js and it’s suitable for shifting all timestamps to the browser’s timezone, which may change automatically as the user travels.

Asking the user to pick their timezone from a dropdown also works and it’s simple to implement, but it’s inconvenient.

An example of timezone identifier is America/Los_Angeles. Pacific Time is the name that locals would recognize, so it would be suitable for display in English. Its offset with respect to UTC is -8 hours in winter and -7 hours in the summer and the exact rules are defined in the IANA timezone database.

Timezone definitions are maintained by the IANA (Internet Assigned Numbers Authority), with official releases a few times a year. Those definitions are installed on Ubuntu under /usr/share/zoneinfo and they’re used by the localtime system call which expresses a time in seconds since 1970 into year, month, day, hour, minutes, and seconds… according to the globally-defined timezone.

Yes, localtime doesn’t take a timezone parameter. In order to format a local time for an arbitrary timezone, the default POSIX way is to set the TZ environment variable, call localtime, then optionally set TZ back to what it was. Environment variables being global to a process, this is not thread-safe, but at least it works without installing a special-purpose library.

In bash/sh you can therefore format the current date in the timezone of your choice as follows:

$ TZ=Africa/Abidjan date
Wed Jul 26 00:15:36 GMT 2017

For a date in RFC3339 format with millisecond precision and GNU date, the following function will work:

$ function date_() { date '+%FT%T.%3N%:z' "$@"; }
$ date_
2017-07-25T17:30:09.528-07:00

Note that date --rfc-3339=ns will almost work, except there’s no ms precision for milliseconds, only seconds or nanoseconds (see man date), and it prints a space instead of the T. ¯\_(ツ)_/¯

Here’s how to obtain the current time in Hong Kong, formatted the way we want:

$ TZ=Asia/Hong_Kong date_
2017-07-26T08:30:24.024+08:00

And this is for formatting “86400 seconds (= 24 hours) after January 1, 1970 UTC”, our current timezone being 8 hours behind UTC at that time of the year of the year on that year:

$ date_ -d @86400
1970-01-01T16:00:00.000-08:00