Sign in to follow this  
Followers 0
tripwire

SensorTag Altitude Logger

16 posts in this topic

I've been playing with the new CC2650 SensorTag over the past few weeks, and am just getting started on my first proper project using it. It uses the onboard BMP280 sensor to measure atmospheric pressure, works out the altitude and continuously logs the data to the external SPI flash.

 

I'm trying out TI-RTOS for the first time which is interesting. It's pretty easy to throw together a proof of concept, but I'm not entirely sure I'm doing things right :)

 

Anyway, I got it working well enough to try some real-world testing over the weekend, so I fitted the coin cell and went for a bike ride. The results were pretty good!

 

Here's a profile of the route as taken from a mapping site (height in m):

 

post-30355-0-34348600-1439246093_thumb.png

 

The red bands are where the altitude drops to zero because the mapping site has no elevation data for a bridge I crossed.

 

Here's what I got after pulling the data from my SensorTag and feeding it into a spreadsheet (again, height in m):

 

post-30355-0-61716000-1439250895_thumb.png

 

The results are a lot better than I was expecting. Measuring altitude from pressure can be inaccurate due to changes in temperature, pressure at sea level and humidity, amongst other things. The dashed line shows the altitude at the start of the trace, which should match up with the end since the route is circular. It's not perfect, but it's only 1.25m out over a period of 150 minutes.

 

All the hills are recognisable, but you can see that my trace appears slanted to the right. That's because I'm taking measurements at intervals of time rather than distance. The climbs look less steep than they should because I'm travelling slower so get more data points per unit distance. Likewise the descents look a bit closer to vertical than they ought to. The top of the first 125m hill is extended because I stopped there for a while and the logger kept going.

 

At some point I'd like to hook it up to a cycle speedometer reed switch so it can record speed and auto-start/stop when moving/stationary. I also want to compress the trace data better: this trip generated about 41kb of data at 4Hz sample rate, so the flash would be full after ~30 hours. Finally I want to try using energytrace on the sensortag, because I'm pretty sure that I don't have it set up properly for low power consumption.

 

bluehash and pine like this

Share this post


Link to post
Share on other sites

This with GPS would be cool.. or you could use the IMU acceleration and integrate it twice.

Nice work!

 

Thanks!

 

Position tracking would indeed be cool, but I think I'm going to stick with distance/speed/altitude logging for this project. That's mainly to keep the power and storage requirements down.

 

I've got a target of 128 hours for the logging duration which will cover eight days (assuming I'm stationary for at least eight hours per day).

 

That works out as about 9 bits per second for the 4Mbit flash on the SensorTag. Currently I'm using about 36 bits per second just for altitude, but I have a plan for getting that down to about 11 and a few viable options to go further from there. I'll post more on that later...

 

I have no idea what the battery life will be at present, or even what's feasible. I'd really like to stick with using a coin cell because that would let me use the case that's included with the SensorTag.

Share this post


Link to post
Share on other sites

@@tripwire This project is neat! I'm just getting started with the CC2650 myself and would really love to see your source code if you're willing to share (in turn I'd of course be developing my project in the open). If not, would you be willing to share some knowledge on how you were able to continuously log to the external 4MB SPI flash? That specifically is where I have been stuck for about a week (I am fairly new to this area); I'd like to just get started continously logging the on-board sensors and see what I can hack together from a longer data set once I can get that going.

 

Any feedback would be super helpful, thanks!

 

Nick

Share this post


Link to post
Share on other sites

@@tripwire This project is neat! I'm just getting started with the CC2650 myself and would really love to see your source code if you're willing to share (in turn I'd of course be developing my project in the open). If not, would you be willing to share some knowledge on how you were able to continuously log to the external 4MB SPI flash? That specifically is where I have been stuck for about a week (I am fairly new to this area); I'd like to just get started continously logging the on-board sensors and see what I can hack together from a longer data set once I can get that going.

 

Any feedback would be super helpful, thanks!

 

Nick

 

Hi Nick,

 

I'm not ready to release the source just yet, but I am planning to write more about this project as it develops. The majority of that will cover the logging system: how the code is structured, the data flow, log data format and the thought process behind it. Hopefully that will be of interest!

 

Regarding your own project, what particular aspect of writing to the flash are you having trouble with? Is it the practical details of communicating with the flash and performing commands, or is it more on the program design side of things? I'd like to hear a bit more as it will help me decide what areas to focus on when I write this up.

 

Rob

Share this post


Link to post
Share on other sites

 

Hi Nick,

 

I'm not ready to release the source just yet, but I am planning to write more about this project as it develops. The majority of that will cover the logging system: how the code is structured, the data flow, log data format and the thought process behind it. Hopefully that will be of interest!

 

Regarding your own project, what particular aspect of writing to the flash are you having trouble with? Is it the practical details of communicating with the flash and performing commands, or is it more on the program design side of things? I'd like to hear a bit more as it will help me decide what areas to focus on when I write this up.

 

Rob

 

Rob,

 

I'm much more interested in the practical aspects of logging to flash at the moment and think that program design from there on up would be more my strong-suit; my first-pass has been to modify the SensorTagApp/SensorTagStack sample firmwares. I'm fairly comfortable now with RTOS on a high-level and have been able to get started with sensor readings. My roadblock is getting this to store persistently, and I've had no luck in scouring E2E forums for others' solutions to this (and I myself haven't been able to glean from the docs/code). The `ext_flash` interface I refer to comes from SensorTag/Board/Devices/, in case you are not familiar with that interface to the Flash. Any call to attempt to open a connection to the flash is unsuccesful for me (`extFlashOpen()`), so I of course can make no read/writes. For me, even a 'Hello World' description to store dummy values would be tremendously helpful for me to get on my way here. If there's anything you can point me towards to that effect in the meantime, I would be very grateful.

 

If you'd be willing to share a relevant snippet around the above issues off the record, that would be appreciated (in case this is preferrable to you, it would still be helpful for me) and would of course be kept offline.

 

Thanks,

 

Nick

Edited by nickb
tripwire likes this

Share this post


Link to post
Share on other sites
Any call to attempt to open a connection to the flash is unsuccesful for me (`extFlashOpen()`), so I of course can make no read/writes.

 

Aha, I think I have just the thing... Here's some information about a similar problem I had when getting the external flash to work in my program:

 

http://forum.43oh.com/topic/8822-warning-jtag-interferes-with-sensortag-external-flash-access/

 

EDIT: I just took a look at the extFlashOpen and it calls extFlashVerifyPart to get the manufacturer/device ID. That will get zeroes when the JTAG is active, so extFlashOpen will return false in that case

 

I also spotted this interesting comment, which might help me cut the power consumption of the flash between write operations :)

 

#ifdef IO_PARKING

// Table of pins to be "parked" in when no device is selected. Leaving the pin

// floating causes increased power consumption in WinBond W25XC10.

nickb likes this

Share this post


Link to post
Share on other sites

It's time for an update. Remember when I said this:

 

I have no idea what the battery life will be at present...

 

When I was writing the firmware I took care to minimise power consumption where possible. The RTOS tasks sleep between measurements and there's no busy-waiting. Writes to the flash are batched up and sent a page at a time, with the flash put in shutdown until the next page is ready. The LEDs and buzzer are not used during logging. I was pretty confident that the power consumption would be acceptable.

 

Well, I finally got around to taking some current measurements two weeks ago and the results were woeful. The average current was somewhere around 12mA (!), so the SensorTag would be draining a fresh CR2032 every 18 hours or so. This was confirmed when the altitude logger cut out after a further 90 minutes of use.

 

Checking the sample projects showed equally bad results, so there had to be something odd going on.

 

I got a tip-off that the latest samples in BLE Stack 2.1 and TI-RTOS 2.14 had better power consumption, which they did to an extent. The SensorTag sample was still drawing 4mA when powered off, however, and that had to be wrong. By now I was getting pretty suspicious of the TMP007 sensor, which had never worked and appeared to have some damage just visible on the (bare) die. Desoldering it fixed the problem, and brought the SensorTag sample's average current down to a few microamps when powered off.

 

Now I had a good sample, but the SensorTag sample is heavily event-based which makes it hard to trace through. It took days (literally!) to work out what the sample was doing differently to my code. Mainly it was down to IO configuration and power profile setup. Most of the components on the SensorTag are configured for low power at startup, but the others take a lot of current. Here's a post giving more detail on that: http://forum.43oh.com/topic/8864-cc2650-sensortag-low-power-checklist/

 

Now I've got the measurement current down to 0.33mA, giving an estimated battery life of 28 days. I expect it can be improved, but for now it's more than good enough!

chicken and bluehash like this

Share this post


Link to post
Share on other sites

I've just got a new storage management system working on the SensorTag, so it's time for another update...

 

As mentioned previously, the altitude measurements are buffered in RAM until there's enough to fill a 256-byte page of the flash memory. The buffered page gets written to the flash and the current page index (stored in RAM) is incremented. That means the altitude log fills successive addresses in the flash memory, starting from zero. That's fine for the first run, but power-cycling would reset the page index and make the program start logging from zero again. To work around this I've been updating a hardcoded log start address in the firmware to start on the next free page. That required reflashing the firmware before every trip. Obviously it would be a lot nicer if the firmware could figure out where to write next without any intervention.

 

Ideally I wanted the firmware to treat the flash as a circular buffer, overwriting old values with new ones when the flash was full. That could be done by storing a couple of page indices alongside the data, updating them as pages are written. It would essentially be a very basic filesystem. Unfortunately it's not as straightforward as I just made it sound. The problem is down to the way flash memory works.

 

Flash memory is a bit peculiar in that the "write" operation is only capable of changing bits from 1 to 0. To change bits from 0 back to 1 requires a separate "erase" operation. That wouldn't be so bad, but it turns out that the "erase" operation can't be performed on individual bytes. Instead the erase operation must be used on a block of multiple bytes (sometimes called an "erase unit"). In the case of the SPI flash on the SensorTag, the smallest erase unit size is 4KB (or 16 pages). Modifying the value of a single byte requires reading the containing 4KB block into RAM, erasing it from flash and then writing back the modified data.

 

The CC2650 has enough memory to cope with this, but it has its downsides. A block erase can take up to 300ms which is an eternity in microcontroller terms. It also takes a lot of power; that's bad in itself but the extra current draw might cause a tired cell's voltage to drop into brownout. Power loss during the erase or write-back could result in corrupted data.That would be bad news if it affected the filesystem metadata.

 

There's various ways around this, but I've settled on a system that avoids the need to modify previously written metadata values. I'm reserving two 4KB blocks for metadata, one of which is active and gets updated when a data page is stored. Approximately 2KB is used to store an allocation table (one byte of metadata for each page of data). When a page of data is written the corresponding byte of the allocation table is set. If the allocation table value is 0xFF then the page is free, otherwise it is in use. I'm using the byte to hold an index identifying the trace that the data page belongs to. Each byte in the table gets written to once as the log expands to fill the whole flash memory.

 

Once all the data pages have been used the logger returns to the beginning of the circular buffer. The oldest block is erased to free some space, which happens every time the free space is used up. Page writes are now recorded in the second metadata block. The metadata blocks each have a sequence number and any values in the newest one override the corresponding values from the older one. Once the second metadata block is full the first is no longer needed, so it can be erased and reused.

 

This arrangement leaves 504KB for data, and the data blocks get erased once only for each 504KB logged. The metadata blocks are erased every 1008KB. None of the metadata values need to be modified in-place, so the read/erase/write-back process isn't required.

bluehash and chicken like this

Share this post


Link to post
Share on other sites

Another way is to use the first byte of each page as an indicator whether a page is already in use. Simply set it to 0 (or any other value than 0xFF) when writing the page. The application then simply has to search for a page that starts with 0xFF to find an empty page.

 

Still needs a mechanism to erase blocks when running out of flash memory.

tripwire and bluehash like this

Share this post


Link to post
Share on other sites

Another way is to use the first byte of each page as an indicator whether a page is already in use. Simply set it to 0 (or any other value than 0xFF) when writing the page. The application then simply has to search for a page that starts with 0xFF to find an empty page.

 

That's a good point; effectively that method would distribute the bytes from the allocation table throughout the whole memory. It would avoid the need to keep two copies of the metadata and neatly sidesteps any edge cases where it could get out of sync with the data. Much simpler!

 

You'd have to be careful to always erase the next block before writing the last page of a block, however. If that's not done the boundary between old and new data would be lost.

 

Also it means that the initialisation code needs to search through the pages looking at the initial bytes to find the start of free space. That's no problem if the data needs to be read anyway, but my firmware just needs to find the end to continue writing. Having the metadata in contiguous blocks means it can be loaded in bulk, which is a bit more convenient.

 

Something I noticed while writing the code is that the first 32 bytes of the allocation tables are free because they map back onto the metadata blocks. I've used eight of those bytes to hold a second-level allocation table. After checking which metadata block is newest the code reads the eight byte table to find the active metadata page. That whole page is read and searched to find the next free page. The result is that the initialisation takes three 8-byte reads and a full page read rather than up to 2048 individual byte reads.

 

With this particular flash chip it would be possible to do something similar for the distributed metadata approach. Rather than reading the first byte of every page you could just check the last page of each erase unit: those where (pageIndex % 16) == 15. If that page is free then check the other pages in the block, otherwise keep going. This works because the blocks are erased completely and fill from their start to their end in order. The final page of the block is always free until all the other pages are in use. That would give a worst-case of 143 byte reads to find the next page.

chicken likes this

Share this post


Link to post
Share on other sites

That's a good point; effectively that method would distribute the bytes from the allocation table throughout the whole memory. It would avoid the need to keep two copies of the metadata and neatly sidesteps any edge cases where it could get out of sync with the data. Much simpler!

 

You'd have to be careful to always erase the next block before writing the last page of a block, however. If that's not done the boundary between old and new data would be lost.

 

Also it means that the initialisation code needs to search through the pages looking at the initial bytes to find the start of free space. That's no problem if the data needs to be read anyway, but my firmware just needs to find the end to continue writing. Having the metadata in contiguous blocks means it can be loaded in bulk, which is a bit more convenient.

 

Something I noticed while writing the code is that the first 32 bytes of the allocation tables are free because they map back onto the metadata blocks. I've used eight of those bytes to hold a second-level allocation table. After checking which metadata block is newest the code reads the eight byte table to find the active metadata page. That whole page is read and searched to find the next free page. The result is that the initialisation takes three 8-byte reads and a full page read rather than up to 2048 individual byte reads.

 

With this particular flash chip it would be possible to do something similar for the distributed metadata approach. Rather than reading the first byte of every page you could just check the last page of each erase unit: those where (pageIndex % 16) == 15. If that page is free then check the other pages in the block, otherwise keep going. This works because the blocks are erased completely and fill from their start to their end in order. The final page of the block is always free until all the other pages are in use. That would give a worst-case of 143 byte reads to find the next page.

 

I wrote a datalogger many years ago that took a different approach. The flash I was using had a few extra bytes per sector for "Out of Band" data. Each sector was something like 264 bytes long, and I used some of the extra bytes for the file indexes. Basically, each sector written had a sequential 16-bit number written to this area that increased by one for each sector write. When the memory wrapped around like a circular buffer, the older records were overwritten.

 

This all works well because when booting the system, the records were scanned and a simple search could locate the point where the last new record was stored because it would be the point where the record number wrapped around. A few precautions were needed because the sector number itself could roll over, but that was simple to take into account. A binary search was also used to make searching a large memory exponentially faster.

 

I recommend you avoid having a file index stored in one area of the flash because you're going to wear that memory out by constantly writing and rewriting to it. There's no wear leveling, and you also have to worry about some flash devices being sensitive to bits flipping if you repeatedly write the same bits to the same sector without an erase. It's best that you spread the indexing out over the entire memory to avoid these issues. With my scheme the log history is retained, and picks up where it left off after resetting or cycling power. I used other bits in the out of band area to mark the beginning and end of log periods, that way the user could look back at previous logs, or just look at the most recent one.

 

It might be easy for you to simply timestamp each record in your log and use that as the index, but you have to bulk erase the flash if the user ever changes the time settings in the clock, which is why I didn't do this.

 

Your project looks interesting and I look forward to when you are able to give out more details. I am a cyclist and this type of project is exciting to see.

tripwire, chicken and dubnet like this

Share this post


Link to post
Share on other sites

I recommend you avoid having a file index stored in one area of the flash because you're going to wear that memory out by constantly writing and rewriting to it.

 

I understand how that would be a problem for a typical file system where you'd need to read/erase/modify/write the index whenever a file was modified.

 

In my case the index is an append-only structure, the same as the logged data. A log that grows from the start until it fills the whole memory would erase and write every data page once. Meanwhile the active metadata block would be erased once, then each byte of the alloaction table would be written once in turn. When the log wraps around to the start the other metadata block is used in the same way. That means the metadata blocks are actually erased and written half as often as the pages that hold the data.

 

[...] and you also have to worry about some flash devices being sensitive to bits flipping if you repeatedly write the same bits to the same sector without an erase.

 

Do you mean overprogramming a flash byte with its current value (ie programming it but not changing the value)?

Share this post


Link to post
Share on other sites

Do you mean overprogramming a flash byte with its current value (ie programming it but not changing the value)?

 

Exactly. I don't remember the part number of the flash I had used at the time, but it was a sector based flash where you would fill a RAM buffer with data and then when it was full, you would flash the entire sector all at once. Since the erased state of the flash was 0xFF in all locations, you could do byte writes by filling the buffer with all 0xFF bytes, and then adding new bytes to it and flashing the entire sector each time an 0xFF was changed to a new data value. It definitely would work that way when I tested it, but the datasheet warned that this kind of write was not recommended and could flip neighboring bits unintentionally.

Share this post


Link to post
Share on other sites

Exactly. I don't remember the part number of the flash I had used at the time, but it was a sector based flash where you would fill a RAM buffer with data and then when it was full, you would flash the entire sector all at once. Since the erased state of the flash was 0xFF in all locations, you could do byte writes by filling the buffer with all 0xFF bytes, and then adding new bytes to it and flashing the entire sector each time an 0xFF was changed to a new data value. It definitely would work that way when I tested it, but the datasheet warned that this kind of write was not recommended and could flip neighboring bits unintentionally.

 

Thanks, I understand the issue now.

 

The flash I'm using has a page-sized RAM buffer too, but it's able to address individual bytes. You can load up to 256 bytes into the buffer and then trigger the write by releasing chip select. At that point the buffered values get written to flash starting at the specified address. There's a restriction that only bytes in the same page as the start address can be written, so the write wraps around if it hits the end of the page. It says that it's possible to program part of a page as long as the current write doesn't overlap the already programmed areas.

 

Beyond that the datasheet is pretty vague. It doesn't have any warnings about cumulative programming time like the flash in the MSP430 MCUs. There's no mention of write-disturb or bit flipping. I even had to go to the winbond website to confirm that the chip was NOR flash rather than NAND... that's not in the datasheet either.

Share this post


Link to post
Share on other sites

Recently I took a trip to the US, which offered a good opportunity to test my altitude logger by recording a profile of the whole journey there. The trace revealed some interesting details about the flights I took, and airline operations in general.

Here's the profile for the entire trip:
 
post-30355-0-23173800-1464732984_thumb.png
 
The x-axis shows elapsed time in minutes. The altitude is shown in metres, measured relative to the start of the trace (not too far above sea level). Despite that I'll be using feet as the unit of altitude here, since that's the standard used in aviation. Because the logger calculates altitude based on air pressure, it is affected by cabin pressurisation. Instead of recording the true altitude of the aircraft it gives a trace of the effective altitude inside the cabin.

The first big peak at the blue cursor is a flight from Edinburgh to London Heathrow. Comparing the cabin altitude trace against real altitude data makes it easier to pick out the main features, so here's a chart showing this flight's altitude as broadcast over ADS-B:
 
post-30355-0-77564500-1464733005_thumb.png
 
And this is a closeup showing what my altitude logger recorded for the same flight:
 
post-30355-0-99639700-1464733027_thumb.png
 
The cursors mark where I think the flight started and finished, based on the fact that the plane was in the air for 70 minutes. From takeoff the pressure falls steadily until the effective altitude in the cabin is about 7000ft, at which point the aircraft is actually at 37000ft. After cruising there for 12 minutes the plane descends and cabin pressure steadily increases.

The cabin pressure reaches ground level before the plane actually lands, so the trace stays flat for the next 12 minutes. In fact, this section of the trace is effectively below ground level while the plane approaches landing. The plane's environmental control system has deliberately overshot and pressurised the cabin to higher than ambient pressure at the destination. At the orange cursor marking the end of the flight you can see a slight increase in altitude. This is when the flight is over and the controller opens the pressurisation valve to equalise with the external air pressure.

It seems this extra pressurisation is done before takeoff and landing to help the system maintain a steady pressure. There's a detailed explanation of the reasons for this here: http://aviation.stackexchange.com/questions/16796/why-is-cabin-pressure-increased-above-ambient-pressure-on-the-ground

Now on to the second flight, which was from Heathrow to Dallas Fort Worth. First the ADS-B trace:
 
post-30355-0-14331200-1464736930_thumb.png
 
And the altitude logger's version of events:
 
post-30355-0-14710700-1464736943_thumb.png
 
Again, the cursors mark the start and end of the flight and line up with the reported duration. The "steps" along the top of the trace match up with changes in cruise altitude from 32000?>34000?>36000ft. Maximum effective cabin altitude is about 5500ft, lower than the first flight even when the lower cruise altitude is taken into account. I think that's down to the use of a newer 777 on the international flight compared to the A319 on the domestic route. Modern planes are increasingly designed to offer lower effective cabin altitudes for passenger comfort.

The stepped flight profile is used to maximise fuel efficiency. Flying higher reduces losses to air resistance, but early in the flight the aircraft is heavy with fuel and climbing is expensive. As the fuel is burned off the optimal cruise altitude increases, so ideally the plane would climb to match. In fact the plane can't climb gradually because modern air traffic control regulations restrict aircraft to set flight levels. The best option under these restrictions is to perform a "step climb" up to a higher level when it's more fuel-efficient than the current one. The flight levels are multiples of 2000ft for flights from the UK to the US, which is why the steps are 32000->34000?>36000ft.

Wrapping up, one of the things I hoped to test by recording this journey was high rates of altitude change. The altitude logger can currently handle rates of change up to

spirilis, yosh, Fmilburn and 4 others like this

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
Sign in to follow this  
Followers 0