Ewon Technical Forum

Full Version: MQTTS - eWON Flexy to Microsoft Azure (Self-signed certificate authentication)
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3 4
Hi Guys,

Here is an working example to connect your Flexy to Azure using the self-signed certificate method :

Code:
Rem --- eWON start section: Cyclic Section
eWON_cyclic_section:
Rem --- eWON user (start)
Rem --- eWON user (end)
End
Rem --- eWON end section: Cyclic Section
Rem --- eWON start section: Init Section
eWON_init_section:
Rem --- eWON user (start)
//################" CONFIGURATION #################
DeviceId$="Flexy205Self"
IotHubName$ ="eWONPROJECT"

Changepushtime% = 2 //Timer to push only Tags that has changed
Fullpushtime% = 20// Timer to push all values
//Select the Tag Group to publish -> 0 or 1
//Tag must be created and at least set in one of the groups.
GROUPA% = 1
GROUPB% = 1
GROUPC% = 1
GROUPD% = 1

// /usr directory operations to do :
// 1. Generate a Self-Signed certificate using
//    openssl req -x509 -sha256 -nodes -days 3650 -newkey rsa:2048 -keyout ReplaceByDeviceID.key -out ReplaceByDeviceID.crt -config openssl.cnf
// 2. Rename the cert and the key with the DeviceID (<DeviceID>.crt and <DeviceID>.key)
// 3. Upload the self-signed cert + the key + the baltimoreCA certificate to the /usr
// 4. Start the script -> You should see some "PUBLISH..." logs in the console.
// 5. Do not forget to select Run > Autorun in order to have the script running at boot

//################"END CONFIGURATION ##############

CLS

//Read number of Tags
NB%= GETSYS PRG,"NBTAGS"
DIM a(NB%,2)

MQTT "Open",DeviceId$,IotHubName$ + ".azure-devices.net"
Mqtt "SetParam","Port","8883"
MQTT "setparam", "log", "1"
MQTT "setparam", "keepalive", "20"
MQTT "setparam", "TLSVERSION", "tlsv1.2"
MQTT "setparam", "PROTOCOLVERSION", "3.1.1"
MQTT "setparam", "cafile","/usr/BaltimoreCyberTrustRoot.pem"
MQTT "setparam", "CertFile","/usr/"+DeviceId$+".crt"
MQTT "setparam", "KeyFile","/usr/"+DeviceId$+".key"
Mqtt "SetParam","Username",IotHubName$+ ".azure-devices.net/"+DeviceId$+"/api-version=2016-11-14"
Mqtt "SetParam","Password","HostName="+IotHubName$+";DeviceID="+DeviceId$+";x509=true"

SETSYS PRG,"RESUMENEXT",1  //Continue in case of error at MQTT "CONNECT"
Mqtt "Connect"
ErrorReturned% = GETSYS PRG,"LSTERR"
IF ErrorReturned% = 28 THEN @Log("[MQTT SCRIPT] WAN interface not yet ready")
SETSYS PRG,"RESUMENEXT",0

ONMQTT "GOTO MqttRx"

//a = table with 2 columns : one with the negative indice of the tag and the second one with 1 if the values of the tag change or 0 otherwise
IsConnected:
//Record the Tag ONCHANGE events into an array.
//Allows to post only values that have changed
FOR i% = 0 TO NB%-1
 k%=i%+1
 SETSYS Tag, "load",-i%
 a(k%,1)=-i%
 a(k%,2) = 0
 GroupA$= GETSYS TAG,"IVGROUPA"
 GroupB$= GETSYS TAG,"IVGROUPB"
 GroupC$= GETSYS TAG,"IVGROUPC"
 GroupD$= GETSYS TAG,"IVGROUPD"
 
 IF GroupA$ = "1" And GROUPA%= 1 THEN Onchange -i%, "a("+ STR$ k%+",2)= 1"
 IF GroupB$ = "1" And GROUPB%= 1 THEN Onchange -i%, "a("+ STR$ k%+",2)= 1"
 IF GroupC$ = "1" And GROUPC%= 1 THEN Onchange -i%, "a("+ STR$ k%+",2)= 1"
 IF GroupD$ = "1" And GROUPD%= 1 THEN Onchange -i%, "a("+ STR$ k%+",2)= 1"
NEXT i%
 
ONTIMER 1,"goto MqttPublishAllValue"
ONTIMER 2, "goto MqttPublishChangedValue"

TSET 1,Fullpushtime%
TSET 2,Changepushtime%
END

//Compute the right time format for AZURE
Function GetTime$()
  $a$ = Time$
  $GetTime$ = $a$(7 To 10) + "-" + $a$(4 To 5) + "-" + $a$(1 To 2) + " " + $a$(12 To 13)+":"+$a$(15 To 16)+":"+$a$(18 To 19)
EndFn

//Publish just the changed tags
MqttPublishChangedValue:
counter% = 0

//Compute JSON
json$ = '{'
FOR r% = 1 TO NB%
IF a( r%,2) = 1 THEN
  a(r%,2) = 0
  negIndex% = a(r%,1)
  SETSYS Tag, "LOAD", negIndex%
  name$= GETSYS Tag, "name"
  json$ = json$ + '"' + name$+ '":"'+STR$ GETIO name$ + '",'
  counter% = counter% +1
ENDIF
NEXT r%
json$ = json$ +    '"time": "'+@GetTime$()+'"'
json$ = json$ +    '}'

IF counter% > 0 THEN
MQTT "PUBLISH","devices/"+DeviceID$+"/messages/events/",json$, 0, 0
PRINT "[PUBLISH ONCHANGE TIMER] " + STR$ counter% + " Tags have changed detected -> Publish"
ELSE
PRINT "[PUBLISH ONCHANGE TIMER] No Tag changes detected! -> Don't publish"
ENDIF
END
 
//publish all tags
MqttPublishAllValue:
counter%=0
json$ =         '{'
  FOR i% = 0 TO NB% -1
      SETSYS Tag, "load",-i%
      i$= GETSYS TAG,"Name"
     
      GroupA$= GETSYS TAG,"IVGROUPA"
      GroupB$= GETSYS TAG,"IVGROUPB"
      GroupC$= GETSYS TAG,"IVGROUPC"
      GroupD$= GETSYS TAG,"IVGROUPD"
     
      IF GroupA$ = "1" And GROUPA%= 1 THEN json$ = json$ + '"' + i$+ '":"'+STR$ GETIO i$ + '",': counter% = counter% +1
      IF GroupB$ = "1" And GROUPB%= 1 THEN json$ = json$ + '"' + i$+ '":"'+STR$ GETIO i$ + '",': counter% = counter% +1
      IF GroupC$ = "1" And GROUPC%= 1 THEN json$ = json$ + '"' + i$+ '":"'+STR$ GETIO i$ + '",': counter% = counter% +1
      IF GroupD$ = "1" And GROUPD%= 1 THEN json$ = json$ + '"' + i$+ '":"'+STR$ GETIO i$ + '",': counter% = counter% +1
     
  NEXT i%    
  json$ = json$ +    '"time": "'+ @GetTime$() +'"'
  json$ = json$ +   '}'
 
  STATUS% = MQTT("STATUS")

 //Is Connected
 If (STATUS% = 5) Then
   Print "[PUBLISH ALL TAGS TIMER] " + STR$ counter% + " tags selected and published"
   MQTT "PUBLISH","devices/"+DeviceID$+"/messages/events/",json$, 0, 0
 Else
   Print "Not connected (" + STR$ STATUS% + ")"
 Endif
End

FUNCTION Log($Msg$)
  LOGEVENT  $Msg$ ,100
  PRINT $Msg$
ENDFN



Rem --- eWON user (end)
End
Rem --- eWON end section: Init Section


I have based my script on the one used in https://techforum.ewon.biz/thread-561.html and I also removed the config page and some extra useless code.


Now with my script, everything is configured at the top of the script :
[attachment=456]

To generate the self-signed certificate and the key for the Flexy, you have to install "openssl" and run the next command :
openssl req -x509 -sha256 -nodes -days 3650 -newkey rsa:2048 -keyout FlexySelfPrivateKey.key -out FlexySelfCertificate.crt -config openssl.cnf

I have also attached a zip file containing files to generate your Flexy certificate and key easily.

This zip contains :
  1. a bat file (generateFlexyCertificates.bat)
  2. a windows version of OpenSSL to run the command easily
  3. a example of certificate you can use for your tests
Exrtact the files somewhere and run the BAT file "[font=Arial]generateFlexyCertificates.bat" to generate them. It will prompt you to enter some personal info. These is pure cosmetic information (Just press ENTER if you do not know).[/font]
At the end you should get two files created : FlexySelfCertificate.crt and FlexySelfCertificate.key

[EDIT] I have now created a webpage that allows you to do that even easier :-)  https://ewonsupport.biz/azurehelper/


Once you get the certificate and key created, you then have to add a new device in AzureIOT hub and select X509 Self-Signed :
[attachment=455]

In the Primary and secondary Thumbprint, you need to copy/paste the certificate Thumbprint.
To get [font=Arial]the thumbprint
, open the certificate with the Windows tool "Crypto Shell Extensions" (double click it should normally work)[/font]
and check the certificate details :

[attachment=453]


[attachment=458]
(Remove the spaces if you have some)

Finally rename the certificate and the key with the DeviceID name and [font=Arial]upload them into the usr/ directory
+ the BaltimoreCACert is still needed
[attachment=457]
and start the script[/font]


[attachment=452]

By the way, somebody has made an excellent video explaining all this (Thanks to him for this great job! :-) ) :



[EDIT 28/7/2022]
Due to the Baltimore CA certificate replacement in Azure IOT Hub (https://techcommunity.microsoft.com/t5/i...-p/2393169), I have updated the CA certificate file that must be used with the script.  This file contains both the current and the new certificate. So, using that CA file will make your script working now and in the future.
For those who are already using the script (and the previous CA file), the new CA file will have to pushed in every Flexy so that they can still work after September 2023
Hi Simon, I copied your script and pasted it into the ewon configuration page. Basic IDE, I changed DeviceId $ and IotHubName $ to the names I use in my account. When I save and execute the script, an error appears on line 49 Mqtt "Connect". I can not connect to Azure, are there other parameters that I need to change?

Edit:
I used the example certificate and now is working correctly, I don't know why It didn't worked with my certificate.

Now is sending data to azure, but I do not know where to see the data that was sent.
Hi,

Good to hear.
I have never managed to see the data in the Azure cloud interface. The only way I found is to use the software "Device Explorer Twin"
https://github.com/Azure/azure-iot-sdk-c...ceExplorer

Simon
once you have device connected to iot hub, you need to configure a streaming analytics job to move the data somewhere else such as an sql database

Hi Simon
can i suggest that given that AZure is major player in your target Iot market that you need to publish a way to use the connection strings etc provided by azure.

as a non programmer this would be really helpful

Hi Simon

i must be doing something wrong, when i try to use ssl it just errors 

it says invalid command 'openssl'; type help for a list
error in openssl
followed instructions

plus changed the host name in line 67 to correct one

will not connect

for some reason the cyclic section would not cut and paste into the IDE, but it does not appear to do anything so i have ignored it
Hi Richard,

You were right about the line 67. I have fixed it.
The cyclic section is indeed not doing anything in the script.

Do you have an update ? Does it still not connect ?

Simon
Hi Simon,

I tried a simplified version of the previous script. However it can never connect properly to the MQTT broker of Azure and it gets stuck with error like "mqtt-Not connected [1DAFF8]". Previous tests showed successful connections on other MQTT brokers without authentication. What can I do to get more information about what is going wrong?

Marc

eWon firmware version 13.2s1
Hi Marc,

To me, there must be something wrong with the certificate or the configuration in Azure.
I have seen once a similar issue and I had to recreate a device in Azure to get it working.

Simon
Hi everyone,

I tried this a couple times with different certificates and kept getting Failed: Mqtt "connect".

The problem was DNS on my 3G modem. Here is a link that helped me out if you're stuck on the same problem:

https://forum.hms-networks.com/t/ewon-fl...il/5523/17
Hi,

This problem is fixed in the firmware 13.2s1

Simon
Hi,

I have problem with certificate, in Azure IoT Hub, I have message "Cet appareil est en cours d'authentification à l'aide d'un certificat X.509.". I already regenerate certificate with generator and Flexy not change state in Azure ? Have you idea ?

So this solution is better ? --> https://techforum.ewon.biz/thread-561.html

Regards
(10-12-2018, 04:28 PM)simon Wrote: [ -> ]Hi Guys,

Here is an working example to connect your Flexy to Azure using the self-signed certificate method :

Code:
Rem --- eWON start section: Cyclic Section
eWON_cyclic_section:
Rem --- eWON user (start)
Rem --- eWON user (end)
End
Rem --- eWON end section: Cyclic Section
Rem --- eWON start section: Init Section
eWON_init_section:
Rem --- eWON user (start)
//################" CONFIGURATION #################
DeviceId$="Flexy205Self"
IotHubName$ ="eWONPROJECT"

Changepushtime% = 2 //Timer to push only Tags that has changed
Fullpushtime% = 20// Timer to push all values
//Select the Tag Group to publish -> 0 or 1
//Tag must be created and at least set in one of the groups.
GROUPA% = 1
GROUPB% = 1
GROUPC% = 1
GROUPD% = 1

// /usr directory operations to do :
// 1. Generate a Self-Signed certificate using
//    openssl req -x509 -sha256 -nodes -days 3650 -newkey rsa:2048 -keyout ReplaceByDeviceID.key -out ReplaceByDeviceID.crt -config openssl.cnf
// 2. Rename the cert and the key with the DeviceID (<DeviceID>.crt and <DeviceID>.key)
// 3. Upload the self-signed cert + the key + the baltimoreCA certificate to the /usr
// 4. Start the script -> You should see some "PUBLISH..." logs in the console.
// 5. Do not forget to select Run > Autorun in order to have the script running at boot

//################"END CONFIGURATION ##############

CLS

//Read number of Tags
NB%= GETSYS PRG,"NBTAGS"
DIM a(NB%,2)

//Start "Try to Connect" timer
ONTIMER 1, "GOTO MqttCONNECT"
TSET 1,10

MqttCONNECT:
MQTT "Open",DeviceId$,IotHubName$ + ".azure-devices.net"
Mqtt "SetParam","Port","8883"
MQTT "setparam", "log", "1"
MQTT "setparam", "keepalive", "20"
MQTT "setparam", "TLSVERSION", "tlsv1.2"
MQTT "setparam", "PROTOCOLVERSION", "3.1.1"
MQTT "setparam", "cafile","/usr/BaltimoreCyberTrustRoot.pem"
MQTT "setparam", "CertFile","/usr/"+DeviceId$+".crt"
MQTT "setparam", "KeyFile","/usr/"+DeviceId$+".key"
Mqtt "SetParam","Username",IotHubName$+ ".azure-devices.net/"+DeviceId$+"/api-version=2016-11-14"
Mqtt "SetParam","Password","HostName="+IotHubName$+";DeviceID="+DeviceId$+";x509=true"
Mqtt "Connect"

//IF No error --> Connected --> Disable Retry timer
TSET 1,0
ONMQTT "GOTO MqttRx"

//a = table with 2 columns : one with the negative indice of the tag and the second one with 1 if the values of the tag change or 0 otherwise
IsConnected:
//Record the Tag ONCHANGE events into an array.
//Allows to post only values that have changed
FOR i% = 0 TO NB%-1
 k%=i%+1
 SETSYS Tag, "load",-i%
 a(k%,1)=-i%
 a(k%,2) = 0
 GroupA$= GETSYS TAG,"IVGROUPA"
 GroupB$= GETSYS TAG,"IVGROUPB"
 GroupC$= GETSYS TAG,"IVGROUPC"
 GroupD$= GETSYS TAG,"IVGROUPD"
 
 IF GroupA$ = "1" And GROUPA%= 1 THEN Onchange -i%, "a("+ STR$ k%+",2)= 1"
 IF GroupB$ = "1" And GROUPB%= 1 THEN Onchange -i%, "a("+ STR$ k%+",2)= 1"
 IF GroupC$ = "1" And GROUPC%= 1 THEN Onchange -i%, "a("+ STR$ k%+",2)= 1"
 IF GroupD$ = "1" And GROUPD%= 1 THEN Onchange -i%, "a("+ STR$ k%+",2)= 1"
NEXT i%
 
ONTIMER 1,"goto MqttPublishAllValue"
ONTIMER 2, "goto MqttPublishChangedValue"

TSET 1,Fullpushtime%
TSET 2,Changepushtime%
END

//Compute the right time format for AZURE
Function GetTime$()
$a$ = Time$
$GetTime$ = $a$(7 To 10) + "-" + $a$(4 To 5) + "-" + $a$(1 To 2) + " " + $a$(12 To 13)+":"+$a$(15 To 16)+":"+$a$(18 To 19)
EndFn

//Publish just the changed tags
MqttPublishChangedValue:
counter% = 0

//Compute JSON
json$ = '{'
FOR r% = 1 TO NB%
IF a( r%,2) = 1 THEN
  a(r%,2) = 0
  negIndex% = a(r%,1)
  SETSYS Tag, "LOAD", negIndex%
  name$= GETSYS Tag, "name"
  json$ = json$ + '"' + name$+ '":"'+STR$ GETIO name$ + '",'
  counter% = counter% +1
ENDIF
NEXT r%
json$ = json$ +    '"time": "'+@GetTime$()+'"'
json$ = json$ +    '}'

IF counter% > 0 THEN
MQTT "PUBLISH","devices/"+DeviceID$+"/messages/events/",json$, 0, 0
PRINT "[PUBLISH ONCHANGE TIMER] " + STR$ counter% + " Tags have changed detected -> Publish"
ELSE
PRINT "[PUBLISH ONCHANGE TIMER] No Tag changes detected! -> Don't publish"
ENDIF
END
 
//publish all tags
MqttPublishAllValue:
counter%=0
json$ =         '{'
  FOR i% = 0 TO NB% -1
      SETSYS Tag, "load",-i%
      i$= GETSYS TAG,"Name"
     
      GroupA$= GETSYS TAG,"IVGROUPA"
      GroupB$= GETSYS TAG,"IVGROUPB"
      GroupC$= GETSYS TAG,"IVGROUPC"
      GroupD$= GETSYS TAG,"IVGROUPD"
     
      IF GroupA$ = "1" And GROUPA%= 1 THEN json$ = json$ + '"' + i$+ '":"'+STR$ GETIO i$ + '",': counter% = counter% +1
      IF GroupB$ = "1" And GROUPB%= 1 THEN json$ = json$ + '"' + i$+ '":"'+STR$ GETIO i$ + '",': counter% = counter% +1
      IF GroupC$ = "1" And GROUPC%= 1 THEN json$ = json$ + '"' + i$+ '":"'+STR$ GETIO i$ + '",': counter% = counter% +1
      IF GroupD$ = "1" And GROUPD%= 1 THEN json$ = json$ + '"' + i$+ '":"'+STR$ GETIO i$ + '",': counter% = counter% +1
     
  NEXT i%    
  json$ = json$ +    '"time": "'+ @GetTime$() +'"'
  json$ = json$ +   '}'
 
  STATUS% = MQTT("STATUS")

 //Is Connected
 If (STATUS% = 5) Then
   Print "[PUBLISH ALL TAGS TIMER] " + STR$ counter% + " tags selected and published"
   MQTT "PUBLISH","devices/"+DeviceID$+"/messages/events/",json$, 0, 0
 Else
   Print "Not connected (" + STR$ STATUS% + ")"
 Endif
End





Rem --- eWON user (end)
End
Rem --- eWON end section: Init Section


I have based my script on the one used in https://techforum.ewon.biz/thread-561.html and I also removed the config page and some extra useless code.


Now with my script, everything is configured at the top of the script :


To generate the self-signed certificate and the key for the Flexy, you have to install "openssl" and run the next command :
openssl req -x509 -sha256 -nodes -days 3650 -newkey rsa:2048 -keyout FlexySelfPrivateKey.key -out FlexySelfCertificate.crt -config openssl.cnf

I have also attached a zip file containing files to generate your Flexy certificate and key easily.

This zip contains :
  1. a bat file (generateFlexyCertificates.bat)
  2. a windows version of OpenSSL to run the command easily
  3. a example of certificate you can use for your tests
Exrtact the files somewhere and run the BAT file "[font=Arial]generateFlexyCertificates.bat" to generate them. It will prompt you to enter some personal info. These is pure cosmetic information (Just press ENTER if you do not know).[/font]
At the end you should get two files created : FlexySelfCertificate.crt and FlexySelfCertificate.key


Once you get the certificate and key created, you then have to add a new device in AzureIOT hub and select X509 Self-Signed :


In the Primary and secondary Thumbprint, you need to copy/paste the certificate Thumbprint.
To get [font=Arial]the thumbprint
, open the certificate with the Windows tool "Crypto Shell Extensions" (double click it should normally work)[/font]
and check the certificate details :





(Remove the spaces if you have some)

Finally rename the certificate and the key with the DeviceID name and [font=Arial]upload them into the usr/ directory
+ the BaltimoreCACert is still needed

and start the script[/font]

I was able to get this working and see the data flowing into Azure Hub but is there any plans to create a script which can Use MQTT over WebSockets so that we do not have to have the clients open 8883 because MQTT over WebSockets works on 443 which is most likely open on the Client side where these devices will be installed.

Also how do you setup the script to run on reboot??

Let me know.
Steve
Hi,

The script is setup autostart. So I have message in Azure IoT Hub on my device " This device is being authenticated with the help of an X.509 certificate."

How many time for check cetificat ?

Best regards
Steve,

No plan to develop MQTT over Websocket at the moment unfortunately.
Hello,

Everything seems to work correctly but I am confused how to see the data.
When one or more tags change in the eWON, I see the following message:

[PUBLISH ALL TAGS TIMER] 4 tags selected and published

In the Device Explorer Twin all looks fine also:
The device has the ConnectionState to Connected and LastConnectionStateUpdatedTime corresponds to the timestamp on which the values have changed.
But in the Data Monitoring, I always see an error:

Receiving events...
Stopped Monitoring events. An error occurred during communication with 'DeviceGateway_2a643097fa88431a8b3e75c911542b3e:ihsuprodamres075dednamespace.servicebus.windows.net:5671'. Check the connection information, then retry.

Can you help me?
How can I be sure that the data is really in Azure because in the activity log of Azure I don't see anything appearing which indicates that values have been saved in the cloud?


Thanks,


Gaëtan
Pages: 1 2 3 4