3rd Party Local API - Beta Testers/Contributers Wanted

[donnyk]

Here is what the output looks like currently, at the end you can see me disarm and alarm via the web interface built into the script. I'll work on a few of the things pounder noted tonight.

Code: Select all

C:\dsc>envisalink2.py
2012-12-05 10:28:41 Alarm Server Starting
2012-12-05 10:28:41 Tested only on a DSC-1616 + EVL-3
2012-12-05 10:28:41 Reading config file
2012-12-05 10:28:42 Envisalink Proxy Started
2012-12-05 10:28:42 Connected to envisalink:4025
2012-12-05 10:28:42 RX < 505 - Login Interaction P: 3
2012-12-05 10:28:42 TX > 005user54
2012-12-05 10:28:42 RX < 500 - Command Acknowledge P: 005
2012-12-05 10:28:42 RX < 505 - Login Interaction P: 1
2012-12-05 10:28:42 TX > 00191
2012-12-05 10:28:42 RX < 500 - Command Acknowledge P: 001
2012-12-05 10:28:42 RX < 610 - Zone 001 Restored
2012-12-05 10:28:42 RX < 610 - Zone 002 Restored
2012-12-05 10:28:42 RX < 610 - Zone 003 Restored
2012-12-05 10:28:42 RX < 610 - Zone 004 Restored
2012-12-05 10:28:42 RX < 610 - Zone 005 Restored
2012-12-05 10:28:42 RX < 610 - Zone 006 Restored
2012-12-05 10:28:42 RX < 610 - Zone 007 Restored
2012-12-05 10:28:42 RX < 610 - Zone 008 Restored
2012-12-05 10:28:42 RX < 610 - Zone 009 Restored
2012-12-05 10:28:42 RX < 610 - Zone 010 Restored
2012-12-05 10:28:43 RX < 610 - Zone 011 Restored
2012-12-05 10:28:43 RX < 610 - Zone 012 Restored
2012-12-05 10:28:43 RX < 610 - Zone 013 Restored
2012-12-05 10:28:43 RX < 610 - Zone 014 Restored
2012-12-05 10:28:43 RX < 609 - Zone 015 Open
2012-12-05 10:28:43 RX < 610 - Zone 016 Restored
2012-12-05 10:28:43 RX < 610 - Zone 017 Restored
2012-12-05 10:28:43 RX < 610 - Zone 018 Restored
2012-12-05 10:28:43 RX < 610 - Zone 019 Restored
2012-12-05 10:28:43 RX < 610 - Zone 020 Restored
2012-12-05 10:28:43 RX < 610 - Zone 021 Restored
2012-12-05 10:28:43 RX < 610 - Zone 022 Restored
2012-12-05 10:28:43 RX < 610 - Zone 023 Restored
2012-12-05 10:28:43 RX < 610 - Zone 024 Restored
2012-12-05 10:28:43 RX < 610 - Zone 025 Restored
2012-12-05 10:28:43 RX < 610 - Zone 026 Restored
2012-12-05 10:28:43 RX < 610 - Zone 027 Restored
2012-12-05 10:28:43 RX < 610 - Zone 028 Restored
2012-12-05 10:28:43 RX < 610 - Zone 029 Restored
2012-12-05 10:28:43 RX < 610 - Zone 030 Restored
2012-12-05 10:28:43 RX < 610 - Zone 031 Restored
2012-12-05 10:28:43 RX < 610 - Zone 032 Restored
2012-12-05 10:28:44 RX < 610 - Zone 033 Restored
2012-12-05 10:28:44 RX < 610 - Zone 034 Restored
2012-12-05 10:28:44 RX < 610 - Zone 035 Restored
2012-12-05 10:28:44 RX < 610 - Zone 036 Restored
2012-12-05 10:28:44 RX < 610 - Zone 037 Restored
2012-12-05 10:28:44 RX < 610 - Zone 038 Restored
2012-12-05 10:28:44 RX < 610 - Zone 039 Restored
2012-12-05 10:28:44 RX < 610 - Zone 040 Restored
2012-12-05 10:28:44 RX < 610 - Zone 041 Restored
2012-12-05 10:28:44 RX < 610 - Zone 042 Restored
2012-12-05 10:28:44 RX < 610 - Zone 043 Restored
2012-12-05 10:28:44 RX < 610 - Zone 044 Restored
2012-12-05 10:28:44 RX < 610 - Zone 045 Restored
2012-12-05 10:28:44 RX < 610 - Zone 046 Restored
2012-12-05 10:28:44 RX < 610 - Zone 047 Restored
2012-12-05 10:28:44 RX < 610 - Zone 048 Restored
2012-12-05 10:28:44 RX < 610 - Zone 049 Restored
2012-12-05 10:28:44 RX < 610 - Zone 050 Restored
2012-12-05 10:28:44 RX < 610 - Zone 051 Restored
2012-12-05 10:28:44 RX < 610 - Zone 052 Restored
2012-12-05 10:28:44 RX < 610 - Zone 053 Restored
2012-12-05 10:28:44 RX < 610 - Zone 054 Restored
2012-12-05 10:28:45 RX < 610 - Zone 055 Restored
2012-12-05 10:28:45 RX < 610 - Zone 056 Restored
2012-12-05 10:28:45 RX < 610 - Zone 057 Restored
2012-12-05 10:28:45 RX < 610 - Zone 058 Restored
2012-12-05 10:28:45 RX < 610 - Zone 059 Restored
2012-12-05 10:28:45 RX < 610 - Zone 060 Restored
2012-12-05 10:28:45 RX < 610 - Zone 061 Restored
2012-12-05 10:28:45 RX < 610 - Zone 062 Restored
2012-12-05 10:28:45 RX < 610 - Zone 063 Restored
2012-12-05 10:28:45 RX < 610 - Zone 064 Restored
2012-12-05 10:28:45 RX < 652 - Partition 1 Armed Mode 0
2012-12-05 10:28:46 RX < 673 - Partition 2 is Busy
2012-12-05 10:28:46 RX < 673 - Partition 3 is Busy
2012-12-05 10:28:46 RX < 673 - Partition 4 is Busy
2012-12-05 10:28:46 RX < 673 - Partition 5 is Busy
2012-12-05 10:28:46 RX < 673 - Partition 6 is Busy
2012-12-05 10:28:46 RX < 673 - Partition 7 is Busy
2012-12-05 10:28:46 RX < 673 - Partition 8 is Busy
2012-12-05 10:28:46 RX < 841 - Partition 1 Trouble LED OFF
2012-12-05 10:28:46 RX < 841 - Partition 2 Trouble LED OFF
2012-12-05 10:28:46 RX < 841 - Partition 3 Trouble LED OFF
2012-12-05 10:28:46 RX < 841 - Partition 4 Trouble LED OFF
2012-12-05 10:28:46 RX < 841 - Partition 5 Trouble LED OFF
2012-12-05 10:28:46 RX < 841 - Partition 6 Trouble LED OFF
2012-12-05 10:28:46 RX < 841 - Partition 7 Trouble LED OFF
2012-12-05 10:28:46 RX < 841 - Partition 8 Trouble LED OFF
2012-12-05 10:28:46 RX < 510 - Keypad Led State - Partition 1 P: 82
2012-12-05 10:32:43 Incoming web connection from ('192.168.99.1', 24789)
2012-12-05 10:32:43 Web request: GET /
2012-12-05 10:32:44 Incoming web connection from ('192.168.99.1', 14196)
2012-12-05 10:32:44 Web request: GET /favicon.ico
2012-12-05 10:33:06 Incoming web connection from ('192.168.99.1', 13074)
2012-12-05 10:33:06 Web request: GET /api/alarm/disarm
2012-12-05 10:33:06 TX > 0401XXXX9C
2012-12-05 10:33:06 Incoming web connection from ('192.168.99.1', 35443)
2012-12-05 10:33:06 Web request: GET /favicon.ico
2012-12-05 10:33:07 RX < 500 - Command Acknowledge P: 040
2012-12-05 10:33:09 RX < 510 - Keypad Led State - Partition 1 P: 81
2012-12-05 10:33:09 RX < 750 - Partition 1 User 0040 Opening
2012-12-05 10:33:09 RX < 655 - Partition 1 Disarmed
2012-12-05 10:33:10 Incoming web connection from ('192.168.99.1', 62696)
2012-12-05 10:33:10 Web request: GET /
2012-12-05 10:33:10 Incoming web connection from ('192.168.99.1', 55991)
2012-12-05 10:33:10 Web request: GET /favicon.ico
2012-12-05 10:33:12 RX < 653 - Partition 1 Ready - Force Arming Enabled
2012-12-05 10:33:20 Incoming web connection from ('192.168.99.1', 17103)
2012-12-05 10:33:20 Web request: GET /api/alarm/arm
2012-12-05 10:33:20 TX > 0301C4
2012-12-05 10:33:21 RX < 500 - Command Acknowledge P: 030
2012-12-05 10:33:21 Incoming web connection from ('192.168.99.1', 3436)
2012-12-05 10:33:21 Web request: GET /favicon.ico
2012-12-05 10:33:22 RX < 510 - Keypad Led State - Partition 1 P: 83
2012-12-05 10:33:22 RX < 656 - Partition 1 Exit Delay in Progress
2012-12-05 10:33:23 Incoming web connection from ('192.168.99.1', 6554)
2012-12-05 10:33:23 Web request: GET /
2012-12-05 10:33:23 Incoming web connection from ('192.168.99.1', 4584)
2012-12-05 10:33:23 Web request: GET /favicon.ico
2012-12-05 10:34:22 RX < 510 - Keypad Led State - Partition 1 P: 82
2012-12-05 10:34:22 RX < 652 - Partition 1 Armed Mode 0

[pounder]

As far as what to filter and what to assume a client will do - I have been programming a long time and whatever you assume would be reasonable is never true, whatever you assume is the worst, there is always something worse that comes up. My advice is keep things as flexible as you can and aim for each client connection acting as close to the master connection as possible since that's the known working situation.


On the other point about installer and master modes, thats one of my primary interests in the module is getting around the need for DLS since quite frankly I think its garbage.

In the short term just being able to do what I do standing at an lcd keypad but do it sitting at my desk, possibly remotely on the wan. In the long term being able to push programming, change codes, schedule mode changes of the panel, etc etc.

Its getting pretty clear that there is going to be more than one "proxy" piece of code for this alarm module so I'd suggest we put our heads together and standardize on the next layer of communication so any front end app built for one of the systems will work on any of them. While that possible now with the port 4025 proxying, I think we all agree that's not human readable enough and needs some data massage to make it simpler for front end apps to be built. I would suggest making a real api rather than just something which can be polled for status.

eg: query zone definitions, query zone status, query user list, query partition list, all the various commands that can be executed at high level - change user code, pgm, arm, disarm, change setting, register and unregister for notifications

Every implementation would not have to support every feature of the api and should just return a not supported code which every front end piece should deal with.

Anyone have any thoughts on this ?

[pounder]

PS - I was going to write my version in C and saw you had started in python so I figured what the heck I will give it a shot. I have done a bit of python coding with plpython for postgres embedded code but never too much outside of that. Mainly C over the years, but I am pretty impressed with how fast I was able to whip something together that does quite a bit of work.

[donnyk]

PS - I was going to write my version in C and saw you had started in python so I figured what the heck I will give it a shot. I have done a bit of python coding with plpython for postgres embedded code but never too much outside of that. Mainly C over the years, but I am pretty impressed with how fast I was able to whip something together that does quite a bit of work.

Pounder,

My initial plan did not include a proxy at all, however one user made a good point that integrating it into an existing environment that is already making use of TPI would require that. Initially my goal was to write an application which would follow all the events and keep a state that was visible over http. Currently the built in web server listens for 5 commands:
-/api/alarm/arm
-/api/alarm/stayarm
-/api/alarm/disarm
-/api/refresh (this one just calls 001, mainly used to generate traffic for debugging)
-/api dumps all the data I have collected out as a json object.

None of this is really finished.. Some sort of authentication needs to be put in place. And as you noted there are way more commands that could be implemented and exposed.

Due to the way the device works though, extending the functionality and doing a proxy is going to have tons of issues. For example, if one proxy client is in installer/maintenance mode, what do you do when another client sends a request? I presume in this case we are going to have to send a 502 w/ the installer code but it will have to come from the proxy rather then the envisalink unit itself. There are plenty of other caveats as well.

I'm typically a C/C++ programmer as well but lately my job has revolved more around php. But the main reason I chose python was the write once run in many locations thing. I've tested and the code I have works just as well on my router with python as my pc.

Have you ever used github before? I was going to post the code there later.

[pounder]

Yeah I know how to use git, have mainly just downloaded snapshots, more familiar with svn to actually work on code as a group.

I would like to hear mikep's comments on this as well since he is also working on something like this.
If there was some way we could unify the api thats exposed between the portal and a version which can run locally that would be even better, it just gives people more choice about how they want to put pieces together for their particular system.

[donnyk]

Yeah I know how to use git, have mainly just downloaded snapshots, more familiar with svn to actually work on code as a group.

I would like to hear mikep's comments on this as well since he is also working on something like this.
If there was some way we could unify the api thats exposed between the portal and a version which can run locally that would be even better, it just gives people more choice about how they want to put pieces together for their particular system.

Pounder,

I just posted what I have so far to github.

https://github.com/juggie/AlarmServer

I'd welcome any questions, comments or attempt to blend what we both have so far together.

I have not used git a lot (mostly used svn) but it is pretty good for this type of project. If you wanted to use some free svn hosting instead I am fine with that as well. Let me know.

[donnyk]

Pounder,

https://github.com/blog/966-improved-su ... nt-support

Seems you can use SVN w/ github as well so if you are comfortable with that then here you go.

[pounder]

Had a quick look at the code, biggest thing I noticed (and I know you have not been taking this into account yet) is there are going to be problems with state handling between clients. I know your plan was just to feed responses back to them all, but picture this situation :

client 1 says arm
client 2 says change backlight intensity
module responds to client1 with a user code required
module barfs on the command from client 2
client 1 sends user code

client 2 never sees an ack or nak to its command and user code never gets there but it sees a failed user message

It definitely needs to have more state and serialization across clients to work correctly under all conditions.

[pounder]

you have a definite typo in code 816 - should be buffer not battery.

[donnyk]

you have a definite typo in code 816 - should be buffer not battery.

I corrected the typo and commited it. You should see that fixed now.

As for the proxying you are right that is not something i took into account but frankly I am not 100% certain how to do it reliably. Some parts are easy, eg the proxy can return the 500:Command Ack response. Other responses not so much..

I thought about blocking a block of time when a command is sent, eg client1 sends a command and for next 10 seconds everything is returned to that client only.. but that wouldn't work, other events could happen in the meantime.

I also thought about if you mapped the expected responses to the commands sent. So that the proxy knew what reply (or type of reply would be expected) to a given response and could map that back to the replying client. That however would not be without it's caveats as well. This would be the option that got us closest to a perfect solution though.

I am not sure there is a 100% reliable solution. The solution may just be to provide one single full TPI client and others with limited functionality.

All this said, proxying TPI is not really a big deal.. what we want to do is create a good api to program against, not rewrite one that has lots of limitations. I think if we provide one full TPI connection which is exactly what users have today that would be fine.

Let me know what your github username is if you have one (or create one) and I will give you write access so you can checkout and commit your changes/fixes back.