Chronograph v1.2 Improvements

Intro

It was quite a while after the initial launch of the Chronograph that I first heard about someone having trouble with the accuracy of the stopwatch and countdown timer.  It was many months later before I heard any complaints about the clock drifting.  It was easy to just brush them off as faulty units.  It happens.  But then emails and messages began to get more and more frequent.  Uh-oh.  We have a problem.

Unfortunately, it's quite rare for the initial release of a product to be absolutely perfect and the Chronograph was certainly no exception.  After swallowing my pride, I dug into the reported issues and took note of the discoveries that were made and the changes necessary to correct them.  This post details that process.  Warning: this post is mostly for super electronics nerds, but there's still a lot of interesting info here for everyone.  Most anyone should be able to understand it on a basic level.

Want the TL;DR version?  Read the last section.  It's pretty concise and gives a good general understanding of the changes.


Primer: How does the Chronograph work?

First, we need to understand a little bit about the system topology and how it all works together.  The Chronograph is made up of three main sub-systems:  a RTC (real-time clock) chip, a display driver, and a uC (microcontroller).  The RTC is dedicated to one very specific task: keeping track of the current time.  After initial setup, this is the part that always knows what time it is.  The display driver is simply a chip that takes in some data and uses that data to determine which portions of the display to light up.  It has some other really cool features, but I won't get into that here.  The uC is the Chronograph's "brain".  This is the part that has to be programmed.  It's also the part that reads the footswitch.  The uC passes data between all the other parts and essentially tells them what to do and when to do it.

As an example, here's what happens when the Chronograph boots up: the uC begins by running setup code.  This is where the "DSGE Chronograph" startup message is contained.  The uC sends that string of text to the display driver, which decodes it and lights up the correct LED segments.  Then the uC asks the RTC what time it is.  The uC translates this info into a format the display driver understands and you now see the current time.

Now that you have a basic understanding of how the Chronograph works, lets dig into the issues.


Issue #1: Timer Accuracy

The most commonly reported issue has been the accuracy of the Chronograph's countdown timer and stopwatch.  It seems that not too many people actually use these modes, therefore it took a while before there were enough people out there using them for me to start getting a steady stream of reported issues.  It turns out both modes had a timing error of about 4%: not good.  It may not sound that bad, but 4% corresponds to an error of 1 minute and 12 seconds for every 30 minutes.  If you're running a timer for a strict set length, you just got cut short!  (Or cut the next band short)

So what was happening?  Well, for the sake of complete transparency, during the testing phase I simply looked at the timer counting up and counting down and thought to myself, "It's counting up and counting down just like it's supposed to!  It works!"  For whatever reason it never occurred to me that I should check its accuracy against another source.  So now I've got a couple hundred of these things out there in the wild and some of them are not even accurate.  Bummer.

Anyways, let's get back on track here.  How do we fix it?   The first step was to take a look at the source code.  After checking and double checking and even triple checking I was confident there was no reason for it to be running fast.  The code clearly said, "start the timer and increment/decrement the seconds value once every second."  But the timing is not right!  What the heck is going on!?

I don't recall how or why this came to me, but I eventually had a revelation.  Microcontrollers have a system clock inside them that ticks away, much like a RTC, except the uC clock is typically MUCH faster, in the order of MHz instead of 32.768kHz.  With many of them you have the option of running the clock internally or using an external oscillator.  Well, fewer parts usually equates to lower cost, so naturally I chose to use the on-board oscillator.  Unfortunately, the built in option is usually less accurate than an external source.

To be fair, this usually isn't a problem because when your uC is executing commands that aren't dependent on real time, who cares if the system clock is running at 8.1MHz or 7.9MHz?  However, it becomes a problem when a timing-critical set of instructions is based on the uC system clock, which is exactly how the Chronograph operated.  Any error in the system clock will be exposed in the timer function.

Now we know the problem and have to decide how to fix it.  There are basically two options: add an external oscillator to the uC to improve the accuracy of the system clock or get the timing source from somewhere else.  If you've ever taken a peek under the hood of the Chronograph you know space is at a premium.  I'd also like to avoid adding the cost of more parts if there's any way to avoid it (adding an oscillator also means adding other external circuitry to stabilize it).  So that leaves one option.  Where else can we get an accurate timing source?

Figure 1: A Typical DS1307 Circuit

Figure 1: A Typical DS1307 Circuit

Let's take a peek at the pins on the RTC chip.  The specific RTC that Chronograph v1.1 and older uses is the popular DS1307.  Here's the datasheet.  We've got pins for a crystal, battery backup, power, communication, and "SQW/OUT".  What the heck is "SQW/OUT"?  Looking further down the datasheet, here's a portion of that pin's description: "When enabled, the SQW/OUT pin outputs one of four square-wave frequencies (1Hz, 4kHz, 8kHz, 32kHz)."  For those that don't know, a hertz (Hz) is a unit of frequency expressed in cycles per second.  So the first available frequency, 1Hz, is one clock cycle per second.  Eureka!  We've got a 1Hz clock source that we know is accurate because it's coming from the RTC, right!?

Wrong.  But we'll get to that later.  It turns out I had already run a PCB trace between the SQW/OUT pin and an extra pin on the uC in case I wanted to use it in the future.  So I did.  After modifying the code to use the RTC source (a minor headache of its own) the stopwatch and countdown timers were running perfectly in sync with any other source I could find.  The amount of error was so low that I couldn't even reliably measure it and I didn't even have to modify the hardware.  Engineering at its finest.  ;)


Issue #2: Clock Accuracy (or the lack thereof)

Although much less common, I was also getting reports from some users that the clock was drifting over time.  Several minutes per month in some cases, but no two users were reporting anywhere close to the same amount of error.  Weird.  Out of curiosity I decided to plug in the final Chronograph prototype that had now been ticking away on the shelf for well over a year.  It was only off by a few seconds.  What gives?

Time to look at the datasheet again.  Noticing any trends?  The Chronograph is based on the DS1307 RTC chip and the datasheet even has a little section titled "Clock Accuracy".  And I DID read it the first time around; promise.  I was very careful to follow the layout guidelines and crystal specifications given.  But there was one little sentence I didn't give much thought to: "Additional error will be added by crystal frequency drift caused by temperature shifts."  Therein lies the answer to all of our woes.

 
Figure 2: "Clock Accuracy" Section of the DS1307 Datasheet

Figure 2: "Clock Accuracy" Section of the DS1307 Datasheet

 

The second time studying the datasheet that sentence caught my attention.  I wonder how much the temperature can affect it?  The datasheet recommends reading Application Note 58 for more info, which I admit I had never done.  Sure enough, right there on page 1, there's a nice little graph:

 
Figure 3:  Typical Frequency vs Temperature for Tuning Fork Crystals

Figure 3:  Typical Frequency vs Temperature for Tuning Fork Crystals

 

What can we learn from this?  Most people on planet Earth live in climates that vary from -40 to +50 degrees Celsius.  Looking at the graph, it appears that the crystal is tuned to hold exactly 32,768Hz at roughly 25 degrees.  Tapering out from there in either direction, it would appear that the worst case scenario of -40 degrees would correspond to a deviation of about -175ppm.  AN58 says an error of 20ppm is equivalent to about 1 minute per month.  Doing some quick math tells us that, worst case, the clock could shift by as much as 9 minutes per month, just because of temperature!

Oh, how I wanted to cuss the person that decided that line in the datasheet wasn't worth bolding and underlining and typing in huge capital letters.  But then I realized: how many clocks are exposed to temperatures that far away from 25 Celsius?  Most of them just sit on our nightstand within the comfort of our homes.  No big deal.  Remember that prototype I mentioned earlier that was only off by a few seconds after over a year of sitting on the shelf?  The temperature of the room that shelf was in only varied by a few degrees over the course of that year.  Thus the error of only a few seconds.  *facepalm*

Of course most guitarists take their pedalboard outside their house at some point.  I'm sure you've already figured it out, but this explains why some users reported pretty significant errors and others didn't.  My prototype unit sat comfortably on the shelf all year.  Everyone else's spent several days in mailboxes, delivery trucks, airplanes, and warehouses.  Then their owners hauled them around in their own cars and so forth.  And then you have to take the changes of seasons and even geography into account.

Knowing what we now know, it doesn't take a genius to realize that all these changes in temperature can easily account for the clock errors that were being reported.  One sentence in a fourteen page datasheet is responsible for the failure of a product.  Please read and re-read the datasheets, folks.  It can cost you dearly.


The Final Solution to Both Issues

It turns out there's one solution to both problems, so I'm combining them into one section.  Since the temperature can affect the accuracy of the RTC, that in turn affects the accuracy of the 1Hz source that's now powering the timers.  So what can we do to eliminate the effect of temperature on the clock?  As luck would have it, some fancy-pants engineers have designed crystal oscillators that have built-in temperature sensors.  The chip knows exactly how temperature affects the frequency.  Essentially, the graph in Figure 3, seen above, is programmed into it.  It measures the temperature every so often and adjusts the oscillator accordingly.  Brilliant!  These devices are called Temperature Compensated Crystal Oscillators, or TCXO's.

Figure 4: The Upgraded RTC Chip

Figure 4: The Upgraded RTC Chip

But wait; there's more!  The fine folks over at Maxim Integrated have taken it a step further and created a RTC chip with a built-in TCXO!  The specific chip I've chosen is called the DS3231 (Figure 4). Take a look at the datasheet.  Can you guess why I picked it?  You didn't actually take the time to compare the datasheets, did you?  I don't blame you.  To make a long story short, the registers that store the time information are exactly the same.  Why is this awesome?  All I had to do was connect the correct pins to the uC and the code I've already written is compatible with our new fancy RTC!  Magical!

How much improvement does this upgrade provide?  The first page of the datasheet specifies:  "Timekeeping Accuracy ±5ppm (±0.432 Second/Day) from -45°C to +85°C"  This translates to a maximum error of about 13 seconds per month, down from about 5 minutes per month!  Considering the majority of users will never use a Chronograph in environments anywhere near those temperature limits, I'd say that's an incredible improvement!

So that's the story of how the poor old inaccurate Chronograph evolved into its super-awesome current form.  The RTC upgrade costs a few dollars more per unit, but I think that's a small price to pay for the massive improvement to the product and the higher rate of happy customers.  :)

Posted on September 19, 2017 and filed under Geek Fuel.