9 Replies Latest reply on Nov 28, 2019 3:21 AM by BigG

    Getting own LoRa data from ttn server

    gypo

      Hi

       

      I hope this reaches someone that can help, I have my fingers crossed so tight they hurt!  After more than a year of trying I'm at the point of giving up and selling my LoRa kit.

      I have designed and build LoRa nodes with sensors with ease and sending data to the ttn servers.  LoRa hardware development for me is going great guns but getting at my data isn't

       

      I thought I saw a way to help vulnerable people around me by designing an expandable system that can help them get the help they need locally & quickly instead of the police not turning up or the ambulance taking an age to respond.  I had mentioned it to several elderly people some who live on their own and they thought it could work for them.  But I was a little optimistic on how easy the coding was.

       

      I have a ttn gateway with an external aerial on the roof of my bungalow and a few sensors (ttn node, arduino LoRa shields and home brews) they all work and can send data to the ttn servers and I can see the data on there.  As I have hit a brick wall I have switched off the gateway and sensors but can have it up and running easily and quickly if someone can help.

       

      I would like to be able to store my data so that I can make use of it and respond to it when the need arises.  I have my own domain and would like to get my data from the ttn servers to a database or other medium that enables me to store it and view it on my domain.

       

      I am OK with designing, building and testing LoRa hardware (electronic stuff) as I have been inventing/designing circuits for many industries for over 50+ years (OAP myself & retired now).  I can write some code but writing code to manage data and connections between web/net related things is beyond me.  I have tried to follow many instructions and videos (on the ttn web sites and around the net) to connect with ttn using MQTT but after the initial part they all get very complicated and either assume I know stuff and skip it, want to use a 3rd party web site or use a raspberry pi.  None of them help.

       

      I have got through quite a few tutorials to the point where I should see my data but either things complain or nothing happens.

       

      What I'm hoping for is help writing the code to get my data from the ttn servers onto either a web page or a database on my domain (or better still both).

      Is there anyone willing to hold my hand for a while to get this going?

       

      Fingers crossed

      Best Regards

      Chris

        • Re: Getting own LoRa data from ttn server
          clem57

          So I will try to direct you, but I need to know a few things first.

          Messages can be transmitted from a gateway to TTN through two protocols: the legacy Semtech UDP protocol, and our new Gateway Connector protocol.

          Which version are you using?

          For the UDP I see a few problems:

          • Message exchange is not reliable, since the exchange is over 2-way UDP.
          • The UDP protocol is based on a format close to JSON, but not fully compliant.

          Are you a single channel gateway?

           

          We do not recommend starting to set up gateways by setting up single-channel gateways. Single-channel gateways are not LoRaWAN-compliant and only offer poor coverage. For this reason, we have chosen to hide single-channel gateways from our gateway map. If you want to start exploring the LoRaWAN world without investing too much, we would rather recommend investing in a Things Gateway or building your own gateway, which are beneficial to the whole community!

           

          If not, What is your gateway of choice or DIY?

          DIY is going to be the toughest to get help with.

           

          Clem

          • Re: Getting own LoRa data from ttn server
            BigG

            I've just picked up on this thread and trying to make sense of your difficulties.

             

            So on one hand, it appears that you have succeeded in your project "I have a ttn gateway with an external aerial on the roof of my bungalow and a few sensors (ttn node, arduino LoRa shields and home brews) they all work and can send data to the ttn servers and I can see the data on there."

             

            But then you have this project goal where you've got stuck and you now need some guidance "I would like to be able to store my data so that I can make use of it and respond to it when the need arises. I have my own domain and would like to get my data from the ttn servers to a database or other medium that enables me to store it and view it on my domain."

             

            So, the issue is getting access to the ttn server from your domain in much the same way as you can "see your data on the ttn server".

             

            Well this is quite common and in order to do this requires the use of an Application Programming Interface (API) between the ttn server and your domain. But, as your own domain is external to the ttn server, the ttn server will of course want to see credentials which match it's requirements. So to start you need to get hold of their "API" documentation so that you can provide the information from you own domain. Now sometimes the ttn server might have an option already available where it "pushes" the data, when it arrives from your nodes, out to a prescribed destination. Other methods may or may not be available too. So, in order for us to help we would need to know who is the ttn server.

             

            Hope that helps.

            2 of 2 people found this helpful
              • Re: Getting own LoRa data from ttn server
                gypo

                Hi

                 

                Thanks for taking an interest in my problem.

                 

                I have reduced the hardware running as I think as soon as I have the data from one sensor onto my domain I should be able to learn from it and add others as I go.

                 

                So for now I have got a 'The Things Gateway' (TTN-GW-868)  and a 'The Things Node' (TTN-ND-868) passing data to the gateway and the gateway passes it on to The Things Network server(s).  I use the The Things Network Console to view the data that my node is sending.

                 

                I hope that answers the question you asked.  The Things Network site has several API's available but that is a black art to me at present, I wouldn't know which to use or what to do with it if I did.

                 

                The hardware I currently have running.

                    The Gateway                                                          The node

                               

                 

                I believe that the ttn server can push the data as their web site gives several options to pass the data from their server to other places.

                 

                I hope that is enough info to enable you to see which way I should go and what I should use.

                If not let me know what you need.

                 

                Just for info;

                If you was to look on the The Things Network Gateway Map I am the only gateway (so far) located in Spondon, Derby in the UK.

                 

                Many thanks again

                Regards

                Chris

                1 of 1 people found this helpful
                  • Re: Getting own LoRa data from ttn server
                    BigG

                    Aha, your explanation is rather helpful. If you can see your data on their  console then you are over the 2/3rds of the way there. The final 1/3 involves creating a "TTN Application".

                     

                    The TTN website has a useful reference. In your case, the "Data API" is what you are after as you want to send the data to your own server: https://www.thethingsnetwork.org/docs/applications/apis.html

                     

                    The generic method they offer for this is MQTT: https://www.thethingsnetwork.org/docs/applications/mqtt/

                     

                    But actually MQTT is not needed in your case.

                     

                    Fortunately, TTN also offer premade "Integrations": https://www.thethingsnetwork.org/docs/applications/integrations.html

                     

                    and buried within "integrations" is an HTTP option: https://www.thethingsnetwork.org/docs/applications/http/

                     

                    That should guide you. Have a read though that then get back to us with any further questions you may have.

                     

                    PS. Don't forget to watch their YouTube video too (just click on the graphic under the header "Getting started video")

                    3 of 3 people found this helpful
                      • Re: Getting own LoRa data from ttn server
                        gypo

                        Thanks for the pointers...

                         

                        But.

                        I have gone round and round on these pages already (I had done before but have taken another look).  I have watched loads of videos from the ttn guys and others but I'm still at a loss of what to do.  I know what I am trying to do I just cant get there, I believe that I'm OK setting up the ttn site to send data out but I don't know how to grab it.

                         

                        I have had HTTP and storage integrations enabled on the ttn site for a long time. But the data from the HTTP integration needs a place to send the data to (hopefully my site but just sending it there doesn't achieve anything yet).

                         

                        That is where it all falls down for me.  The video examples send data to a third party site where you can see it but I'm trying to get the data onto my own domain (web site/pages or database) not on a third party site.  Being able to see it on the ttn or a third party site doesn't let me store it locally or act on it if I need to.  I'd rather have the data direct from the ttn site rather than via yet another site.

                         

                        I'd be appreciative of help getting my domain (web page(s) / database)  set up to receive the data from the HTTP integration... 

                        I'm sorry but I don't have a clue how to do that.

                         

                        Best Regards

                        Chris

                          • Re: Getting own LoRa data from ttn server
                            BigG

                            Hi Chris

                             

                            Not to worry. I suspected that this might be the case as the real work is developing the code on your own domain (web hosting site) and sometimes the webhosting sites have restrictions on server to server communication, which requires some setup to get it to work. Also, most web hosting sites offer PHP as part of the preinstalled "LAMP" or "WAMP" server packages so you would need to know PHP scripting if this is what you want to do.

                             

                            I can give you a hand if you like as there are a number of simple things we could do quickly to helping you get what you want. Send me a direct message (you may need to connect or follow me in order to do so) if you like.

                             

                            Kind regards

                             

                            Colin