Project Reboot - A major Change Direction

I have given up on accurate estimation of sunrise (or dawn) and sunset (or dusk) directly from daylight measurements. It seems easy but it is not, and cloud cover blows you away!

Full Daylight Model Fitting

Okay thats the new path, fit the full daylight model. The solver was a major coding hurdle but it is done now. If I feed the solver calculated LDR readings (quantitized to be fair) it has no problems solving latitude and longitude directly to within a few km (and the average cloud factor). A few hundred metres if the LDR reading are not quantized). If the data has consistent light reduction due to cloud cover, then no problem. It cloud cover is inconsistent (between morning and afternoon) then it will be a problem, you can't have everything! A better cloud cover model can be used but that is an optimisation for later.

Although the TLS2591 would be ideal as it has higher resolution (16 bit) than the internal ADC (10 bit), the LDR and the internal ADC can be used. I calculate a 10k load resistor would be approprate (for my LDR). Here is the LDR ADC reading (red-right) versus Daylight (blue-left) Lux level used by my solver:

The solver is based on the Nelder-Mead "ameoba method", but any steepest descent solver will work fine. Here is the inner core of the function (i.e. minmising Error2), the input variables to be optimised are CloudFactor, Lat and Lon:

  Error2=0.0;
  for (i=0;i<SampleSize;i++) {
    LT=SampleData[i][0]/60.0;
    SampleVoltage=(double)SampleData[i][2];

    // Determine GMST, RA and Dec
    GST=GMST(JD,LT-LTC);
    SunData(JD,LT-LTC,&RA,&Dec,&EoT);

    // Local Hour Angle and Altitude
    LHA=15.0*(GST-RA)+Lon;
    Alt=degrees(asin(sin(radians(Dec))*sin(radians(Lat))+cos(radians(Dec))*cos(radians(Lat))*cos(radians(LHA))));
    
    // Daylight intensity (Lux) and Sensor Voltage
    Lux=DayLight(Alt)*(1.0-CloudFactor);
    SensorVoltage=floor(1023.0/(1.0+10000.0/20000000.0*pow(Lux,0.75))+0.5);

    //Calculate Least Square Fit error
    Error2=Error2+(SensorVoltage-SampleVoltage)*(SensorVoltage-SampleVoltage);
  }

Okay back to the Box code to include the new solver.

Ported the code to the Box (Arduino) and a day of debugging. Using test data the Box reports a latitude within 30 km (longitude almost exact). No surprises that the latitude is off as it is the "hardest" to measure and calculate. Still using float, trials with double showed no improvement. The limitation is the "shallow bottom" of the search space and the effect of ADC quantisation.

I am using the sum of absolute values rather than sum of squares because (1) the sum range is too great for float (at least initially so not really an issue) and (2) I want less weight to cloud and ADC quantisation noise.

Interestingly enough if I use the square root of the absolute value then I get a very good result, within 1.3 km. I don't know if this is repeatable? Not convinced.

Next is to rework the menu and sample storage code.

Okay, the code has been rewritten and the Box is now collecting data. Hopefully no bugs.

First Data

The first data was off by 400 km. Not a great result. I am also getting some data corruption, perhaps it is time for a new RTC module. Anyway here is what the data looked like:

The yellow line is the fitted model. In my mind it is a pretty good fit to the data (blue). The morning was cloudy and the afternoon clear. Also the sensor location is near a south-west facing window. The heel in the blue data would be direct sunlight on the sensor box in the afternoon. So okay, not the best box location. I could do much better.

Two parameter SunFactor

I reworked SunFactor for differences bewteen moring and afternoon sun light:

The fit is quite good and the positional error is down to 120 km. So really the next step is modifying the light sensor to be less sensitive to direct sunlight and putting the box in a better location.

The solver is semi-stable. It needs a better initial estimate than 0 degrees latitude to get going (reliably).

Fix the initial solver estimate by using the sunrise and sunset equations (as previuosly developed). The solver then improves these for variations in sunlight intensity (due to site and cloud cover effects).

Back to the transition model

Well, what can I say I am back looking at the twilight transition model! The full daylight model requires the sensor to me placed in a non-obstructed location. Not that easy, so I reworked the model:

The current location has a better balance between morning and afternoon sunlight but is still affected by direct sunlight in the mid-afternoon. I matched a model 15 minutes before and after sunrise/sunset (-4 to 2 degrees altitude). This time I only scaled the modelled LDR load resistor. The result was within 170 km of the true location.

Upon reflection on the model, it seems to be the right place as the Lux curve (red) has an identifiable inflection points (circled red), that are sunrise and sunset.

Clearly if I ensure sunrise and sunset are treated equally (by the box location) then I should get better results. I am going to put a diffuser over the sensor to see it that helps as well.

Added the pingpong diffuser. Took a while to workout how to do it:

Why a diffuser? Without a diffuser the sensor will read 100% when point at the source but ~0% when pointer perpendicular. This could be corrected in software but better if the sensor is omni-directional. The pingpong ball is about the best I could think off.

Magic