Strange readings

[Mr-HaleYa]

I am using
esp32 ttgo sim 800l (https://rb.gy/ogywnq)
Dfrobot dht22 (https://www.dfrobot.com/product-1102.html)

my code

DHTNEW DHT(18);
DHT.setWaitForReading(true);

float DHTtempBuffer[7];
float DHThumidityBuffer[7];
int DHTReadings = 7;             // number of DHT readings before choosing a median

void getDHT(){
  for (int i = 0 ; i < DHTReadings; i++) {
    DHT.read();                                     // Gets a reading from the DHT sensor
    DHThumidityBuffer[i] = DHT.getHumidity();
    DHTtempBuffer[i] = DHT.getTemperature();
  }
  sortData(DHThumidityBuffer, DHTReadings);                     //sorts array low to high
  DHThumidity = DHThumidityBuffer[(DHTReadings - 1) / 2];       //gets median of the array
  sortData(DHTtempBuffer, DHTReadings);                         //sorts array low to high
  DHTtemperature = DHTtempBuffer[(DHTReadings - 1) / 2];        //gets median of the array
}

So my problem is that even tho I use a filtering algorithm to get rid of highs and lows by taking 7 readings then choosing a median I will occasionally get some massive dips in temperature (and spikes in humidity as a result). my temperature in my room stays about 24°C and the dips are -12 and 12 °C so half the temp and half * -1. I have no clue why it is doing this... It runs every ~2 mins and when the bad readings start it does it consistently for a few runs before correcting itself.

Nothing moves nothing is touched nothing changes... It just goes wack... Could it be a library error? I don't see how it could be but you're a smart man @RobTillaart so you may see something I'm missing. I have also thought of the potential that its a dud sensor. I hope not bc i ordered 20 more before testing the one I'm using. I had the thought that maybe adding more reading (like 15 instead of 7) so there is more to data to filter but it looks like all the readings it's taking in those fault zones are bad...

Would appreciate any suggestion - Regards Hale

[RobTillaart]

The strange numbers seem to indicate that some bits are not read correctly. 12 equls 8+4 so 2 bits.
Most often this is caused by the (missing) pull up resistor. So three questions:
How long is the wire between ESP and DHTsensor?
What is the value of the pull up resistor?
What is the voltage level of the powersupply?

[Mr-HaleYa]

Thanks for the quick reply,

like I said initially I am using a DFrobot DHT22

1. How long is the wire between ESP and DHT sensor?

I am using the provided 12in cable, the wiki says the sensor is good up to 20m cable (I have my doubts on this tho)

2. What is the value of the pull-up resistor?

I found this comment about the product

The PCB has an I2C pull-up resistor and a decoupling cap

Additionally, all diagrams I have found of it wired show it without external pullups and all the docs I have read say it works with esp32. Also a direct reply from the company said that no external pull-up was needed due to the one on the PCB.

3. What is the voltage level of the power supply?

I have it connected to the 5v rail of the esp as the doc says it is 5v powered. I do have it being powered of the front ports of my desktop tho... could this all be attributed with it being supplied limited current?

[RobTillaart]

1. 14 inch ~ 35 cm, normally I would add a pull up resistor 4K7
   Do you have a scope to see the squareness of the signals on the line?
   Very useful tool when doing hardware.

2. Strange that the documentation mentions an I2C pull up as the DHT22 is not a I2C device.

The fact that all diagrams don't show a pull up can be caused by the fact that they use short wires?
Check this tutorial - https://learn.adafruit.com/dht/connecting-to-a-dhtxx-sensor
it definitely uses a pull up.
Also the equivalent datasheet - https://akizukidenshi.com/download/ds/aosong/AM2302.pdf
promotes a pull up.

3. The 5V should work, although it might give 5V on the ESP pins and I do not know if these can stand it for longer periods. But you have too many good readings so this is not the prime suspect.

4. The DHT.read() function returns an error code if it detects one.
   4a. You do not check on this, why not?
   4b. Can you create a log file to see the error code when the dips occur.
   4c. If errors are detected, adjust the code so it retries after e.g. 5 seconds.

5. Can you try this project? it seems to be an ESP specific library.
   https://www.dfrobot.com/blog-910.html

[Mr-HaleYa]

1. Do you have a scope to see the squareness of the signals on the line?

sorry.. I do not have an o-scope since the university has closed

2. Strange that the documentation mentions an I2C pull up as the DHT22 is not an I2C device.

this was not documentation, it was a reply a user left on a purchase.

The fact that all diagrams don't show a pull up can be caused by the fact that they use short wires?

The tutorials (plural) that are on their site all show them using the same cable that is shipped with it.

Check this tutorial - https://learn.adafruit.com/dht/connecting-to-a-dhtxx-sensor
it definitely uses a pull up.
Also the equivalent datasheet - https://akizukidenshi.com/download/ds/aosong/AM2302.pdf
promotes a pull up.

That DHT 22 is not on a PCB and is instead just the raw sensor and yes the raw sensor very much does require a pull up between VCC and data, but the DFrobot version is the raw sensor mounted to a PCB and the PCB has a 10k pull-up between VCC and data just as shown in that example.

4. The DHT.read() function returns an error code if it detects one.

Isn't it also supposes to through a crazy number at you (-999 if I remember) when it gets an error and we have none of those?

5. Can you try this project? it seems to be an ESP specific library.

that library is just a fork of the Arduino-DHT library so I didn't think it would be any different, I also wanted to use yours lol. However, It does say

none of the DHT libraries I found were written to work without errors on the ESP32. For ESP32 (a multi core/ multi processing SOC) task switching must be disabled while reading data from the sensor.

he is specifically talking about the dht22 so this might actually be legit

[RobTillaart]

Thanks for your replies

3. pull up
   The fixed 10 K pull up on the PCB might be too high so an addition 4K7 might improve the signal quality.

4. DHT.read() return value
   I used a crazy number so it would not be mistaken for an "valid temperature or humidity"
   -999 is crazy enough for that purpose. The return value can see if there is e.g. an CRC error
   and that helps to solve the issue.

5. interrupts
   The DHTNEW library has a flag to disable interrupts during the time critical data extraction
   it is pretty new and I have only tested it on an UNO. PLease give it a try.

Add this line before DHT.read()

DHT.setDisableIRQ(true);

With my next purchase-round I'm going to add a number of ESP32 so I can do more testing.

[Mr-HaleYa]

so I powered up the project again to see if it would reoccur and sure enough, it did. However, I have found something interesting. When I press the reboot button my readings stay stable for quite some time before doing it ~8 passes. If I unplug the sensor and plug it back in I get 1-3 good passes then it rapidly drops to -12 after 2-5 passes it bumps to 12 and after another 2-5 int recovers. What could this mean???

[Mr-HaleYa]

side note: I changed it to 15 reading instead of 7 to see if it would help but no it has not

red is unplugged and replugged and yellow are just power cycles with onboard button

so in my debug log when I get the dips it looks like this (humid, temp°C)

40.40	25.70   //it always gives 1 good reading as far I can see
100.00	12.90
100.00	-12.90
100.00	-12.80
100.00	12.80
100.00	12.90
100.00	-12.80
100.00	-12.90
100.00	12.90
100.00	-12.90
100.00	-12.90
100.00	-12.90
100.00	-12.90
100.00	-12.90
100.00	-12.90

on a good pass, the readings look more like

40.50	25.60
41.70	25.70
40.70	25.70
40.60	25.60
40.70	25.70
40.70	25.70
40.60	25.60
40.60	25.60
40.60	25.60
40.60	25.60
40.60	25.60
40.70	25.70
40.60	25.70
40.60	25.60
40.70	25.70

In the passes right before it fails it looks like

40.60	25.60
41.60	25.70
40.50	25.60
40.50	25.60
40.60	25.70
40.70	25.80
40.60	25.70
40.50	25.60
40.60	25.70
40.50	25.60
40.60	25.70
40.60	25.70
100.00	12.80
40.60	25.70
100.00	12.80

[RobTillaart]

Further analysis
The fact that there are so much good readings, means that the code flow is OK.
Hardware issues has been discussed and you need to test and verify these.

In the software to read the DHT sensors the timing is the most critical part. For the much faster ESP32 (compared to UNO) this timing should not be a problem however the "multitasking" behavior of the ESP has been mentioned as a possible cause in this and other timing issues. However differences in hardware including differences in quality of the sensors (including fake and cheap clones) have caused many problems in the past too.

In this analysis I will prove the cause is not in the software but in the hardware, and that a work around can possibly be made in software.

Humidity
Jumps to 100.0 in the code the value is constrained to the range 0..100 so the value 100 will not inform us what happend with the received bit-pattern. Come back to this later.

Temperature
I see three temperature values in your log, so lets analyse the underlying bit pattern. In this analysis the MSB is most informative.

temperature = ((_bits[2] & 0x7F) * 256 + _bits[3]) * 0.1;
...
if (_bits[2] & 0x80)  // negative temperature
  {
    temperature = -temperature;

reversing these lines of code gives us

Value	Val * 10	bits[2] pattern	bits[3] pattern
25.70	257	0000 0001	0000 0001
12.80	128	0000 0000	1000 0000
12.90	129	0000 0000	1000 0001
-12.90	-129	1000 0000	1000 0001

Notice : the Most Significant Bit has shifted 1 place to the left.

cause negative sign
This means that the most right bit of bits[2] is in fact the LSB of the real humidity value.
as a LSB is relative noisy this fluctuates between 0 and 1. That causes the negative sign to pop up randomly when this error occurs.

cause high humidity
As the left shift moves in a bit in bi[0] which is the MSB of the humidity, it adds 6553.6 to the humidity, causing it to be >> 100 and therefor it will be constrained to 100.

AS far as I can see all wrong readings have a humidity of 100. So we might assume that when shift happens, we always shift in a HIGH bit. Where does that come from and WHY?

Cause of the shift
In the code in _readsensors the bits are defined by the length of a LOW period measured in micros. These lines (around 170-180)

  loopCnt = DHTLIB_TIMEOUT;
    while(digitalRead(_pin) == HIGH)
    {
      if (--loopCnt == 0) return DHTLIB_ERROR_TIMEOUT;
    }

    if ((micros() - t) > 40)
    {
      _bits[idx] |= mask;        <<< sets HIGH bit
    }

As we get an extra HIGH bit we must conclude the time is larger than 40 us.

The number 40 is based upon the specs in the datasheet of the DHT sensors:

• 26-28us ==> 0
• 70 uS ==> 1

40 us is 42% larger than the 28us
40 us is 42% smaller than the 70us
So the margin is in both directions equal in size in relative terms (harmonic average). That is the reason I choose 40. As this core code worked for years with many clients it is a good choice.

So for some reason the data pin stays HIGH too long (sometimes), or has difficulties to become LOW fast enough. An detailed analysis of the hardware is mandatory to find the cause.

CONCLUSION
The root cause is not found in the code, it works too well too often. The above analysis shows that the data-pin does not goes LOW fast enough according to the specs / datasheet.

A possible solution to work around this incorrect behavior is to adjust the timing threshold value in the software, in concreto the value in line 176 from 40 to 50.
This gives the signal 10 us extra time to get LOW and is still far enough from the 70 to be discriminating between 0 and 1. If the problem still occurs at 50, one could even try 60, but it must stay between 28 and 70.

Please test if this proposal solves the issue

[Mr-HaleYa]

You are amazing... You literally broke it down and explained it so well. How do I buy you a coffee???

Anyways, I was on board with the idea that it was a hardware issue not explicitly software related. I modified the dhtnew.cpp file and changed it to 50 and so far not a single bad reading. either your a freaking wizard and solved it or it just hasn't reoccurred yet. I have restarted, unplugged, and power cycled like 20 times, and it's going strong.
all rows look like this

18.50	24.50
13.10	24.90
13.20	24.90
13.20	24.90
13.30	24.90
13.40	24.90
13.40	24.90
13.40	24.90
13.30	24.90
13.30	24.90
13.20	24.90
13.20	24.90
13.20	24.90
13.20	25.00
13.20	25.10

I will keep you informed on the status of the next 24 hours. Might I suggest that if it does not falter again meaning this minuscule increase in time was able to eliminate the problem, you should make this an update to prevent future people from suffering as I (briefly) did?

Thank you for all the time you have put into your responses - Hale

[RobTillaart]

Thanks for the compliments, No wizard otherwise I would focus on other things than software.
It is just (hard) analytical thinking and understand the details.

I still think it is a hardware issue somehow as the timing is out of specification.
Fortunately it is an problem that can be solved in software in a way it has very little chance to affect any other platform as 50 is well under the 70 uS.

I will prepare a PR with the adjusted threshold and wait for the results of your 24h test.

[Mr-HaleYa]

dang it...

100.00	-11.60
100.00	-11.60
100.00	-11.70
100.00	11.60
100.00	-11.60
100.00	11.70
100.00	11.70
100.00	-11.60
100.00	-11.60
100.00	-11.60
100.00	11.70
100.00	11.70
100.00	11.70
100.00	11.60
100.00	11.60

that's with 50... The lower numbers are bc its colder inside today.

100.00	11.70
100.00	11.80
100.00	11.80
100.00	11.70
100.00	11.80
100.00	11.80
100.00	11.80
100.00	-11.80
100.00	11.80
100.00	-11.80
100.00	11.80
100.00	11.80
100.00	11.80
100.00	11.80
100.00	11.80

and that's with 60....

100.00	-11.90
100.00	-421.40
100.00	-421.40
100.00	-421.40
100.00	-421.40
100.00	-421.40
100.00	421.40
100.00	421.40
100.00	421.40
100.00	-421.40
100.00	421.40
100.00	-421.40
100.00	-421.40
100.00	-421.40
100.00	-421.40

I tried 30 just for fun and got this ^^^