Switch to desktop version  
MQTT - Create a messages buffer when connection to the broker is down - Printable Version

+- Ewon Technical Forum (https://techforum.ewon.biz)
+-- Forum: Development (https://techforum.ewon.biz/forum-50.html)
+--- Forum: BASIC Script (https://techforum.ewon.biz/forum-52.html)
+--- Thread: MQTT - Create a messages buffer when connection to the broker is down (/thread-1048.html)



MQTT - Create a messages buffer when connection to the broker is down - simon - 17-10-2019

Hi,

Some of you use MQTT connection to send Tag live values to different IOT platforms.

However, you may also want to buffer the messages in case the MQTT connection is down.

As of firmware 14, you now have the possibility to detect the MQTT disconnection from the BASIC and JAVA.

So, the idea here is to create a function in my script (here we will do it in BASIC) that saves the messages when the connection is down and create another function that sends these messages when the connection is up again.

Here is the script (based on the https://ewonsupport.biz/mqtt broker).
(!! This script only works with firmware 14 and > !!)

Code:
Rem --- eWON start section: Cyclic Section
eWON_cyclic_section:
Rem --- eWON user (start)
Rem --- eWON user (end)
End
Rem --- eWON end section: Cyclic Section
Rem --- eWON start section: Init Section
eWON_init_section:
Rem --- eWON user (start)

MsgDBPath$ = "/usr/mqttmsg_db.txt"
MaxMsgBufferFileSize% = 5000000 //max size of file used to save msg in bytes

TopicUsed$ = "/topic/flexy/data"

SETSYS INF, "LOAD"
SerNum$ = GETSYS INF, "SERNUM"

//Configure MQTT Connection parameters
MQTT "OPEN", SerNum$ , "ewonsupport.biz"
MQTT "SETPARAM", "PORT", "1883"
MQTT "SETPARAM", "KEEPALIVE", "10"
MQTT "SUBSCRIBE","/cmd/flexy",2

//Launch the MQTT process in the background
SETSYS PRG,"RESUMENEXT",1  //Continue in case of error at MQTT "CONNECT"
MQTT "CONNECT"
ErrorReturned% = GETSYS PRG,"LSTERR"
IF ErrorReturned% = 28 THEN LOGEVENT "[MQTT SCRIPT] WAN interface not yet ready" ,-1
SETSYS PRG,"RESUMENEXT",0

MsgId% = 0
Last_ConnStatus% = -1


//When receiving a message from Broker, "GOTO MqttReceiveMsg"
ONMQTT "GOTO MQTTRECEIVEMSG" 

//WAIT 3 Seconds before starting to send data
ONTIMER 1, "GOTO SENDDATA"
TSET 1,3
END

SENDDATA:
TSET 1,0 //Stop the timer
//Read MQTT Connection Status ( 5 = Connected, other values = Not connected)
ConnStatus% = MQTT "STATUS"

IF Last_ConnStatus% <> ConnStatus% THEN
  IF ConnStatus% = 5 THEN //Connection is back online
    LOGEVENT "[MQTT SCRIPT] Flexy connected to Broker",100
    @SendSavedEvents() //Parse Saved Messages and publish them
  ELSE
    LOGEVENT "[MQTT SCRIPT] Flexy disconnected from Broker",1
  ENDIF
  Last_ConnStatus% = ConnStatus%
ENDIF

//Compute Message
MyMSG$ = STR$ MsgId% + " - Hello From Flexy " + SerNum$
IF ConnStatus% = 5 THEN //If connected --> Publish
  MQTT "PUBLISH", TopicUsed$ , MyMSG$, 2,0
  PRINT "Message published to the MQTT broker"
ELSE //If not connected --> Save message in file
  @SaveMQTTEvent(MyMSG$)
ENDIF
MsgId% = MsgId% + 1

TSET 1,5 //publish every 5 seconds
END

MQTTRECEIVEMSG:
//Executed when receiving messages from Broker
   MessageQty%=Mqtt "READ"  //Return the number of pending messages
   IF (MessageQty%>0) Then
      MsgTopic$= MQTT "MSGTOPIC"
      MsgData$ = MQTT "MSGDATA"
      LOGEVENT "[MQTT SCRIPT] Message '"+ MsgData$ + "' received on topic " +MsgTopic$,100
      GOTO MQTTRECEIVEMSG
   ENDIF
END

FUNCTION SaveMQTTEvent($Msg$)
$filesize% = FS "size", MsgDBPath$
IF $filesize% < MaxMsgBufferFileSize% THEN
   OPEN "file:" + MsgDBPath$ FOR BINARY APPEND AS 1
   PUT 1, $Msg$ + CHR$(10)
   CLOSE 1
   LOGEVENT "[MQTT SCRIPT] Flexy not connected - event saved" ,-1
ELSE
   LOGEVENT "[MQTT SCRIPT] Flexy not connected - event not saved, buffer file is full" ,-1
ENDIF
ENDFN

FUNCTION SendSavedEvents()
  $FileExist% = FS "isFile", MsgDBPath$
  IF $FileExist% = -1 THEN
    RETURN
  ENDIF
 
  $NbrMsg% = 0
  OPEN "file:" + MsgDBPath$ FOR BINARY INPUT AS 1
  $Line$ = ""
  $ReadNext:
  IF EOF 1 THEN GOTO $ReadDone
  $CHAR$ = GET 1,1
  IF $CHAR$ = CHR$(10) THEN
    MQTT "PUBLISH",TopicUsed$ , $Line$, 2,0
    $Line$ = ""
    $NbrMsg% = $NbrMsg% + 1
  ELSE
    $Line$ = $Line$ + $CHAR$
  ENDIF
  GOTO $ReadNext
  $ReadDone:
  CLOSE 1
  LOGEVENT "[MQTT SCRIPT] " + STR$ $NbrMsg% + " saved messages published",100
  ERASE MsgDBPath$
ENDFN


Rem --- eWON user (end)
End
Rem --- eWON end section: Init Section


PS : Be careful that if you use a big buffer file (See MaxMsgBufferFileSize% variable), the Flexy could take some minutes to push all the messages back to the broker when the connection is up again.  During this time, no "live" messages will be pushed or stored.

If you have some questions, let me know here under

Simon


RE: MQTT - Create a messages buffer when connection to the broker is down - JohnCockerill_jke - 22-11-2019

Hello,
I'm currently working on a project with the objective to push data from a flexy205 (with 4G Modem) to azure.
I firstly tried your demo to push data to your mqtt broker: https://ewonsupport.biz/mqtt but the flexy can't connect to the broker..
Message in console : Operation failed : MQTT "CONNECT"
Do I have to activate something in the config of the flexy ?

Firwmare version : 13.0s0


RE: MQTT - Create a messages buffer when connection to the broker is down - simon - 25-11-2019

Hi,

This is most probably because the TCP port 1883 (Or 8883 if you use MQTT over TLS) is blocked on your Internet connection.

PS : I also advise you to upgrade you Ewon to the last official version.

Simon


RE: MQTT - Create a messages buffer when connection to the broker is down - JohnCockerill_jke - 02-12-2019

Hello Simon,

Just made the upgrade of the firmware.
Checked with the operator to see if port 1883 with the used sim card (didn't get answer yet) but also tried another sim card type (M2M).

Still not able to connect to the broker of the example (I used the exact code of your example).

any idea ?


(25-11-2019, 11:55 AM)simon Wrote: Hi,

This is most probably because the TCP port 1883 (Or 8883 if you use MQTT over TLS) is blocked on your Internet connection.

PS : I also advise you to upgrade you Ewon to the last official version.

Simon

(02-12-2019, 11:44 AM)JohnCockerill_jke Wrote: Hello Simon,

Just made the upgrade of the firmware.
Checked with the operator to see if port 1883 with the used sim card (didn't get answer yet) but also tried another sim card type (M2M).

Still not able to connect to the broker of the example (I used the exact code of your example).

any idea ?


(25-11-2019, 11:55 AM)simon Wrote: Hi,

This is most probably because the TCP port 1883 (Or 8883 if you use MQTT over TLS) is blocked on your Internet connection.

PS : I also advise you to upgrade you Ewon to the last official version.

Simon

OK just found the solution in another post of the forum (DNS Config) :

kpb

Junior Member

Joined: Sep 2017
Threads: 0
Posts: 1
Reputation: 0

#9
 07-04-2019, 09:59 PM

Hi everyone,

I tried this a couple times with different certificates and kept getting Failed: Mqtt "connect".

The problem was DNS on my 3G modem. Here is a link that helped me out if you're stuck on the same problem:

https://forum.hms-networks.com/t/ewon-fl...il/5523/17



RE: MQTT - Create a messages buffer when connection to the broker is down - simon - 02-12-2019

Hi,

Yes, indeed ! Well done!

This bug is fixed into the firmware 13.2s1 and >

Simon