Hacking an analog clock to sync with NTP - Part 5


This is how it looks after I have put everything together. The Arduino sketch is available here.

The 2 jumper wires soldered to the clock mechanism are connected to pins D0 and D1 on the ESP-12 (in any order). When the device first boots up, it presents an access point which can be connected to via the PC or smartphone. Once connected, the captive portal redirects the web browser to the configuration page:

  

A custom field has been added to the WiFi configuration page to enter the current clock time in HHMMSS format. Try to set the clock time to as close to the current time as possible using the radial dial at the back of the clock so the clock will have less work to do catching up.


In the config page, the HTML5 Geolocation API is also used to obtain your current location (so if your web browser asks if you would like to share your location, answer "yes").

This is then passed to the Google Time Zone API to obtain the time and DST offset of your time zone. This allows us to convert the UTC time obtained via NTP to the local time.

Clicking "Save" will make the device connect to the configured AP. Once that is successful, the configuration portal is disabled. The next time the device boots up, it will automatically connect to the configured AP without starting the portal. If you wish to start the portal instead, simply connect pin D0 to GND before starting up the device. The program will force start the configuration portal if it detects that D0 is low.

The following videos show the ESPCLOCK in action. If the clock time is behind NTP time, it will fastforward to catch up:


If NTP time is behind clock time by a couple of minutes, the clock will pulse and wait for NTP time to catch up:


With every tick, the clock time is persisted to the EEPROM so even if the device loses power, it will not lose the current time setting, unless that is changed using the radial clock dial, in which case the configuration portal should be used again to enter the new clock time.

Unfortunately, this clock can only be practically used when connected to the mains. When connected to a 2400mha battery, it lasted slightly more than a day (27.5 hours to be exact). A typical analog quartz clock will run for a year or more on a single AA battery!

Part 1 - Part 2 - Part 3 - Part 4

Comments

  1. Hi, I was attempting to compile the code from https://github.com/victor-chew/espclock 04 Nov 2016

    I received an error code
    clock00:140: error: 'D1' was not declared in this scope


    I found the following libraries need to be added to Arduino 1.6.11
    WifiManager
    https://github.com/tzapu/WiFiManager

    ArduinoJson.h
    https://github.com/bblanchon/ArduinoJson

    esp8266/Arduino
    https://github.com/esp8266/Arduino/tree/master/libraries

    Thanks

    ReplyDelete
    Replies
    1. Well...
      having previously added the (assumed) needed additional modules noted above...

      I added the code below to define the input output ports, just after..
      const char* GOOGLE_API_CERT = "AD:B8:13:99:64:F5:75:F5:78:5C:FA:43:19:EA:8F:AF:2B:AE:54:FE";

      Add something similar selecting the pins you want to use:
      static const uint8_t D0 = 0; // AP mode configuration
      static const uint8_t D1 = 1; // Select a pin (for coil, I think)
      static const uint8_t D2 = 2; // Select a pin (for coil, I think)
      // In effect this is contrary to the instructions in Pt 5 Para. 1

      Hope this helps others..

      Delete
  2. I understand that EEPROM has a limited number of write cycles (usually about 100000) before it fails. If you persist the time every second, wouldn't that mean that you'll have exhausted the EEPROM's lifespan in just over a day (28 hours or so)? Or am I completely mistaken about how EEPROM works?

    ReplyDelete
  3. I was just looking at same and identifying if there might be an issue.

    I came back to see if the author is monitoring the post.... at least once a week. Perhaps the author is on holiday.

    Do post what you discover. I would have thought the EEPROM update might occur a lot less frequently. If it was once an hour that would be an life of around 11 years assuming only 100000 writes. I need to find some time to look at the specification for the devices EEPROM.

    P.S. you might want to look carefully at the pins used. I eventually went with:
    static const uint8_t D0 = 14; // Set one of the reset input to D5 (pin 14)
    static const uint8_t D1 = 13; // D7
    static const uint8_t D2 = 15; // D8
    On a LoLin developer module with ESP8266 on it.

    ReplyDelete
  4. Sorry for the late reply. I was indeed on holiday with the family.

    This is indeed a problem. Thanks for pointing it out. Originally, I updated the clock time every minute, but even then it means the lifespan of the EEPROM is extended by 60.

    There are a few workarounds. One is to stop writing the clock time to the EEPROM, which means every time the clock loses power, one has to set it up again.

    The other might be to hook up a more write-resistent EEPROM module, but that increase the cost and complexity of the project.

    I wonder if there's some kind of wear-leveling algorithm that can be used to extend the lifespan of the EEPROM writes?

    ReplyDelete
  5. I found the discussion here instructive:
    http://www.esp8266.com/viewtopic.php?f=32&t=7106&start=4

    "A better implementation would be to use the SPI flash directly, and only perform an erase, when the 4kB sector is full. You can do that by prefixing your data with a "magic byte" (!= 0xFF), so you can search for the last occurance of that magic byte within the sector and append new data without performing an erase. If you write 8bytes every 10s, a flash sector should last 16years that way."

    Will investigate this approach when I have more time...

    ReplyDelete
  6. Hi, I tried to build one and finally succeed.

    At first, it returns "HTTP Return -1", I changed the function convertLocalTime() when "HTTP GET failed" then set ntpTime manually, I wonder is that GOOGLE_API_CERT not available?
    How can I get a GOOGLE_API_CERT my own?

    Second, it seems will crash somehow, perhaps too many debug information... need more examination.

    Thank you share this project. :D

    ReplyDelete
    Replies
    1. OK, I got crash information
      Exception (29):
      epc1=0x4000e1b2 epc2=0x00000000 epc3=0x00000000 excvaddr=0x00000004 depc=0x00000000

      ctx: cont
      sp: 3fff1ec0 end: 3fff25e0 offset: 01a0

      >>>stack>>>
      ....TL; DR
      <<<stack<<<

      ets Jan 8 2013,rst cause:2, boot mode:(3,6)

      load 0x4010f000, len 1264, room 16
      tail 0
      chksum 0x0f
      csum 0x0f
      ~ld
      Reset request detected
      Loaded config file
      loc = 22,120
      readClockTime: 01:32:02
      *WM: settings invalidated
      *WM: THIS MAY CAUSE AP NOT TO START UP PROPERLY. YOU NEED TO COMMENT IT OUT AFTER ERASING THE DA

      Delete
  7. GOOGLE_API_CERT is not used at all. I will remove it from the Github source code. Initially I thought it was required, but even after removing it, the timezone call works, so I left it out.

    The HTTP call sometimes returns -1, but subsequent calls should work. Leave it running for a while longer to see how it goes.

    ReplyDelete
    Replies
    1. Umm, well anyway
      if one uses...
      F3:4D:26:17:92:3A:EA:40:EF:31:C3:A0:D8:7D:BF:64:A6:F2:93:26
      It will work, as of 16 Nov 2016

      I also created my own GoogleAPI key and added it to the end of the GOOGLE_API_URL
      i.e.
      const String GOOGLE_API_CERTS = "abcdefghijklmop_mWhGR9K6Bg7Cu2cfjICRW1k"; // See https://developers.google.com/maps/documentation/timezone/intro
      // Obviously the above key changed, as its specific to one user. Reading the Google documentation details why

      const String GOOGLE_API_URL = "https://maps.googleapis.com/maps/api/timezone/json?location=[loc]&timestamp=[ts]&key=[ke]";

      [ke] is my Google API key.
      Where the replacements are done in the code for [loc] and [ts] I added a replacement for [ke]
      url.replace("[ke]", GOOGLE_API_CERTS);

      'Hey presto' it now seems to work as intended..

      Delete
    2. There appear to be at least two certificates with the most recent coming live on the 10th November 2016. This would mean storing the two certificate fingerprints then allowing for a fingerprint update every ~84 days. I assume they are coming from different servers from Google (unless my traffic is being subverted ;-) ). Your next update will be end Jan and or beginning of February.

      I am sure they have good reasons for structuring the scheme this way and I shall dig deeper for to a better understanding of their rationale.

      In the near term think I will have the user set a timeozone in the web setup screen and leave this rather interesting issue until another cold wet and windy day.

      Delete
  8. I am able to call the API without GOOGLE_API_CERTS. Am I missing something?

    ReplyDelete
  9. Perhaps the servers I am assigned from Europe have different certificates.
    Though tonights is 216.58.198.174 in California USA apparently.
    Still could be looking at my ip address
    Here are the keys I recovered..
    //const char* GOOGLE_API_CERT =
    "AD:B8:13:99:64:F5:75:F5:78:5C:FA:43:19:EA:8F:AF:2B:AE:54:FE";
    const char* GOOGLE_API_CERT =
    "F3:4D:26:17:92:3A:EA:40:EF:31:C3:A0:D8:7D:BF:64:A6:F2:93:26";
    // 16:42:50:C1:0E:AC:90:00:F4:A3:0E:B9:0D:DB:B8:BC:59:B1:74:3C

    I pasted this into a web browser and then checked the certificate
    https://maps.googleapis.com/maps/api/timezone/json?location=51.4826,0.0077&timestamp=1479251603
    Perhaps I am doing it incorrectly.

    ReplyDelete
  10. P.S. here is how I checked to see what the certificates were originally (and if perhaps they were being intercepted).
    https://www.grc.com/fingerprints.htm

    ReplyDelete
    Replies
    1. If it's of interest here is how I have fixed my issue with the TLS authentication against the Google server...

      I have switch from
      HTTPClient http;
      http.begin(url.c_str(),GOOGLE_API_CERT);
      etc..
      To
      // Use WiFiClientSecure class to create TLS connection
      WiFiClientSecure client;
      ...

      The latter is drawn directly from the HTTPSRequest sample in the 'HTTP over TLS (HTTPS) example sketch'

      The essence of this is that I always get a response for the Google time zone request regardless of certificate. Wiring can then decide what to do with the response valid certificate or otherwise. The code can then elect to proceed to handle the received fingerprint and data as the code sees fit. Code can record and allow the fingerprint to be updated as necessary and present the key in a web form for user verification. Perhaps I can push it to www.grc.com for independent verification as well.

      I also seem to have the original code crash when the fingerprints don't match. This did not happen with the random miss match I tested against using the example method.

      My purpose now diverges significantly from yours as I wanted to use the basis your time zone management on an ESP8266. My system will then push a temperature measurement at a Raspberry PI every 10 minutes into a database, extending the capability of my Evohome Raspberry Pi central heating system by monitoring the heating and hot water pipe temperatures using multiple custom constructed, hardened, DS18B20 temperature sensors. Other room temperatures will rapidly follow. The timezone management will synchronise all the temperature measurements to every 10 minutes (I could just use the NTP minutes and seconds without time zone correction, but wanted to get it all working). Battery operation is planned for later by shutting down for 10 minutes and resending a temperature.

      I hope you have found the feedback interesting.

      Thanks so much for posting the example.

      Delete
  11. So far so good, I use Li-Po 3.7v 3400mAh battery as power source and it could provide power about 7 days, and I wonder is there a way to save power to get more battery life?
    Also, can it connect last wifi ssid automatically and allow change adjust clock's time(without going to ap mode)?

    ReplyDelete
    Replies
    1. > I wonder is there a way to save power to get more battery life?

      That's the million dollar question. There are some interesting comments at the project page: https://hackaday.io/project/16742-espclock, but none I find feasible. It would be quite easy to switch to letting the quartz crystal drive the clock (and let the ESP12 deep-sleep), but knowing what's the clock time upon waking up is the big problem.

      > can it connect last wifi ssid automatically and allow change adjust clock's time

      If only there's some easy and power-efficient way to read the current clock time...

      Delete

Post a Comment

Popular posts from this blog

Adding "Stereo Mixer" to Windows 7 with Conexant sound card

Roomba navigation algorithm