Solarmax Inverters Data to MQTT

Posted by: Mauro 3 years ago

(0 comments)

Some 12 years ago we installed solar panels at home, and connected them to two Solarmax inverters: a 3000S and a 6000S.

Both can be connected to an Ethernet network, and both have an additional RJ-45 socket for serial communication. This serial connection can be used to connect the two inverters to each other, so that with a single Ethernet connection both can be polled for real time and historic production data.

The network protocol had been reverse engineered, allowing the use of simple scripts instead of the proprietary MaxTalk (Windows-only) program. The page that helped back then is still online here: https://2007.blog.dest-unreach.be/2009/04/15/solarmax-maxtalk-protocol-reverse-engineered/

I decided to port the original perl script to python, and managed to build a Linux desktop application using wxPython, which nicely presented the data read from the inverters... when the inverters were on, i.e. during the day only (after sunset, they go to sleep).

Today, with an MQTT broker, InfluxDB and Grafana already running on a cloud VPS, I decided to dig up the old python script and add the glue to get the inverters data displayed in Grafana, making them available even during the night!

The data pipeline is as usual (for more details check the other posts of this blog):

1. A python script talks to the inverters, generating a JSON string.
2. The JSON string becomes the payload of an MQTT message, and gets published to an MQTT broker, using a new topic (e.g. "solarmax").
3. An MQTT input node of a Node-RED flow subscribes to the topic, and injects the JSON objects as time-series data into an InfluxDB.
4. Finally Grafana connects to InfluxDB and graphs the data.

On to the python scripts (which would need refactoring... but happen to work anyway :-)

solarmax.py

import socket, re, sys

resp_ptrn = re.compile('\{..;..;..\|..:(.*)\|(....)\}')
stat_cmd_ptrn = re.compile('DD..|DM..|DY..')
elog_cmd_ptrn = re.compile('EC..')

logcodes = { '20002' : 'Irradiazione insufficiente',
             '20003' : 'Avvio',
             '20004' : 'Funzionamento su MPP',
             '20005' : 'Ventola attiva',
             '20006' : 'Regime a potenza massima',
             '20007' : 'Limitazione temperatura',
             '20008' : 'Alimentazione di rete',
             '20009' : 'Limitazione corrente continua'
           } 

def date_decode(s):
  d = int(s[-2:], 16)
  m = int(s[-4:-2], 16)
  y = int(s[:-4], 16)
  return '%04d-%02d-%02d' % (y, m, d)

def time_decode(s):
  tsec = int(s, 16)
  h = tsec / 3600
  m = (tsec % 3600) / 60
  sec = tsec % 60
  return '%02d:%02d:%02d' % (h,m,sec)

def ecxx_print(s):
  sli = s.split(',')
  if len(sli) == 4 and sli[3] == '0' :
    tstamp = date_decode(sli[0]) + ' ' + time_decode(sli[1])
    try:
      edescr = logcodes[str(int(sli[2], 16))]
    except:
      edescr = str(int(sli[2], 16))
    return tstamp + ' - ' + edescr
  else :
    sys.stderr.write('Unsupported response: ' + s + '\n')
    return s

def pac_print(s):
  w = int(s, 16)
  return str(w/2)

def kwh_print(s):
  k = int(s, 16)
  return '%0.1f' % (k/10.0)

def kwt_print(s):
  k = int(s, 16)
  return str(k)

def typ_print(s):
  if s == '4E34' : return '3000S'
  elif s == '4E48' : return '6000S'
  else : return 'Unknown device'

def stat_print(s):
  sli = s.split(',')
  if len(sli) == 4 :
    d = date_decode(sli[0])
    while d[-3:] == '-00' :
      d = d[:-3]
    k = kwh_print(sli[1])
    p = pac_print(sli[2])
    h = kwh_print(sli[3])
    return d + ': ' + k + ' kWh, ' + p + ' Wmax, ' + h + ' h'
  else :
    sys.stderr.write('Unsupported response: ' + s + '\n')
    return s

cmdd = {
         'CAC' : ['Numero accensioni', kwt_print, '#'],
         'PAC' : ['Potenza AC', pac_print, 'W'],
         'PDC' : ['Potenza DC', pac_print, 'W'],
         'PIN' : ['Potenza installata', pac_print, 'W'],
         'KDY' : ['Energia prodotta oggi', kwh_print, 'kWh'],
         'KLD' : ['Energia prodotta ieri', kwh_print, 'kWh'],
         'KLM' : ['Energia prodotta il mese scorso', kwt_print, 'kWh'],
         'KLY' : ['Energia prodotta l\'anno scorso', kwt_print, 'kWh'],
         'KMT' : ['Energia prodotta nel mese', kwt_print, 'kWh'],
         'KYR' : ['Energia prodotta nell\'anno', kwt_print, 'kWh'],
         'KT0' : ['Energia totale prodotta', kwt_print, 'kWh'],
         'KHR' : ['Tempo totale acceso', kwt_print, 'h'],
         'TYP' : ['Tipo apparato', typ_print, '']
       }

def crc16(s) :
    sum = 0
    for c in s :
        sum += ord(c)
    sum %= 2**16
    return '%04X' % (sum)

def crc_check(s) :
    crc_c = crc16(s[1:-5])
    crc = s[-5:-1]
    if crc_c == crc :
        return 1
    else :
        return 0

def request_string(devaddr, qrystr) :
  msglen = 19 + len(qrystr)
  hexlen = '%02X' % (msglen)
  reqmsg = 'FB;' + devaddr + ';' + hexlen + '|64:' + qrystr + '|'
  crc = crc16(reqmsg)
  final_reqmsg = '{' + reqmsg + crc + '}'
  return final_reqmsg

def response_to_pstringli(resp) :
  m = resp_ptrn.match(resp)
  if not m : return ['']
  if not crc_check(resp) : return ['']
  rli = m.group(1).split(';')
  retli = []
  for rr in rli :
    if rr == '' :
      retli.append('')
      continue
    lr = rr.split('=')
    if lr[0] in cmdd.keys() :
      pstr = cmdd[lr[0]][0] + ': ' + cmdd[lr[0]][1](lr[1]) + ' ' + cmdd[lr[0]][2]
    elif stat_cmd_ptrn.match(lr[0]) :
      pstr = lr[0] + '> ' + stat_print(lr[1])
    elif elog_cmd_ptrn.match(lr[0]) :
      pstr = lr[0] + '> ' + ecxx_print(lr[1])
    else :
      rrli = lr[1].split(',')
      if len(rrli) == 1 :
        pstr = rr + ' (dec: ' + str(int(lr[1], 16)) + ')'
      else :
        pstr = str(rrli)
    retli.append(pstr)
  return retli

def response_to_desc_value_unit(resp) :
  m = resp_ptrn.match(resp)
  if not m : return [[]]
  if not crc_check(resp) : return [[]]
  rli = m.group(1).split(';')
  retli = []
  for rr in rli :
    if rr == '' :
      retli.append([])
      continue
    lr = rr.split('=')
    if lr[0] in cmdd.keys() :
      dvu = [ cmdd[lr[0]][0], cmdd[lr[0]][1](lr[1]), cmdd[lr[0]][2] ]
    elif stat_cmd_ptrn.match(lr[0]) :
      dvu = [ lr[0] + '> ' + stat_print(lr[1]), '', '' ]
    elif elog_cmd_ptrn.match(lr[0]) :
      dvu = [ lr[0] + '> ' + ecxx_print(lr[1]), '', '' ]
    else :
      dvu = [ '', str(rrli), '' ]
    retli.append(dvu)
  return retli


def response_to_value(resp) :
  m = resp_ptrn.match(resp)
  if not m : return [[]]
  if not crc_check(resp) : return [[]]
  rli = m.group(1).split(';')
  retli = []
  for rr in rli :
    if rr == '' :
      retli.append([])
      continue
    lr = rr.split('=')
    if lr[0] in cmdd.keys() :
      dvu = cmdd[lr[0]][1](lr[1])
    elif stat_cmd_ptrn.match(lr[0]) :
      dvu = lr[0] + '> ' + stat_print(lr[1])
    elif elog_cmd_ptrn.match(lr[0]) :
      dvu = lr[0] + '> ' + ecxx_print(lr[1])
    else :
      dvu = str(rrli)
    retli.append(dvu)
  return retli

class SMConn(object) :
  def __init__(self, ipaddr, tcpport, debug=0) :
    self.connected = 0
    self.recvbufsize = 1400
    self.debug = debug
    self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    try:
         self.sock.connect((ipaddr, tcpport))
    except:
         sys.stderr.write('Could not connect to Solarmax device.\n')
         return
    self.connected = 1

  def send(self, s) :
    self.sock.sendall(s.encode())
    if self.debug : sys.stderr.write('SMConn sent: ' + s + '\n')

  def receive(self) :
    s = self.sock.recv(self.recvbufsize)
    if self.debug : sys.stderr.write('SMConn recv: ' + s + '\n')
    return s.decode()

  def close(self) :
    self.sock.close()
    self.connected = 0

solarmax-json.py

#!/usr/bin/python

import time
import json
import subprocess
from solarmax import *

ipaddr = "192.168.1.123"
tcpport = 12345

try:
  spout = subprocess.check_output(["ping", "-c", "1", "-W", "1", ipaddr],
                                  stderr=subprocess.STDOUT)
except subprocess.CalledProcessError:
    print('Solarmax IP is not responding.')
    exit(1)

smc = SMConn(ipaddr, tcpport)
if not smc.connected : exit(1)

class SMDev(object) :
  def __init__(self, name, address) :
    self.name = name
    self.adr = address
    self.pac_now = 0
    self.kwh_today = 0
    self.data_today = [] # kWh, Wmax, h ?
    self.dd_slist = [] # last days production
    self.dm_slist = [] # last months production

  def get_past_data(self, smc) :
      smc.send(request_string(self.adr, 'DD01;DD02;DD03;DD04;DD05'))
      response = smc.receive()
      pstrli = response_to_pstringli(response)
      self.dd_slist = pstrli
      smc.send(request_string(self.adr, 'DD06;DD07;DD08;DD09;DD10'))
      response = smc.receive()
      pstrli = response_to_pstringli(response)
      self.dd_slist += pstrli
      smc.send(request_string(self.adr, 'DM01;DM02;DM03;DM04;DM05'))
      response = smc.receive()
      pstrli = response_to_pstringli(response)
      self.dm_slist = pstrli

  def get_current_data(self, smc) :
      smc.send(request_string(self.adr, 'PAC;KDY;DD00'))
      response = smc.receive()
      # print(response) # raw data string
      # pstrli = response_to_pstringli(response)
      pstrli = response_to_value(response)
      if len(pstrli) == 3 :
        self.pac_now = pstrli[0]
        self.kwh_today = pstrli[1]
        self.data_today = pstrli[2][6:]

  def print_current_data(self) :
      print('Instant AC Power [W]: ' + str(self.pac_now))
      print('Energy today [kWh]: ' + str(self.kwh_today))
      print('Data today: ' + str(self.data_today))

  def print_current(self) :
      print('-------------------------')
      print('Device: ' + self.name + ' (addr: ' + self.adr + ')')
      self.print_current_data()
      print('')

  def get_current_dict(self, smc) :
      cdict = {}
      self.get_current_data(smc)
      cdict['name'] = self.name
      cdict['addr'] = self.adr
      cdict['pac'] = self.pac_now
      cdict['kwh_today'] = self.kwh_today
      return cdict


devli = [SMDev('3000S', '03'), SMDev('6000S', '06')]

outli = []
for dev in devli :
    # dev.get_current_data(smc)
    # dev.print_current()

    devdict = dev.get_current_dict(smc)
    outli.append(devdict)
    # json_obj = json.dumps(devdict, indent=4)

json_obj = json.dumps(outli)
print(json_obj)

For the python code to run without exceptions in python3, I had to encode() and decode() the string passed and received from the socket (see the send() and receive() methods in solarmax.py).

solarmax-json.py outputs a json list of solarmax device attributes:

$ python3 solarmax-json.py
[{"name": "3000S", "addr": "03", "pac": "613.0", "kwh_today": "13.1"}, 
{"name": "6000S", "addr": "06", "pac": "1460.0", "kwh_today": "27.6"}]

Interestingly, in python3 (3.7.3) the Pac values are represented as floats (with a .0 appended), while python2 (2.7.16) outputs them as integers.

The order of attributes also differs:

$ python solarmax-json.py
[{"pac": "497", "name": "3000S", "kwh_today": "13.2", "addr": "03"}, 
{"pac": "1242", "name": "6000S", "kwh_today": "27.8", "addr": "06"}]

solarmax-json.py first checks if the inverter is alive, using ping. If it's not, it just exits with a non-zero error code.

The script terminates with an exit(1) if pinging the IP of the inverter fails, or a connection to it cannot be made.

$ ./solarmax-json.py
Solarmax IP is not responding.
$ echo $?
1

The shell script to run the python code, check its exit value, and publish monitoring data via mqtt:

$ cat solarmax-mqtt-pub.sh
#!/bin/bash
SMAXJSON=$(/home/pi/solarmax/solarmax-json.py)
[ $? -eq 0 ] && mosquitto_pub -h <mqtt_server> -t "solarmax" \
    -m "$SMAXJSON" -p 8883 \
    -u "<mqtt_user>" -P "<mqtt_password>" --capath /etc/ssl/certs/

Script invoked by cron every minute:

$ cat /etc/cron.d/solarmax-mqtt-pub
# run every minute to send mqtt msg with solarmax inverters telemetry
* *  * * *   pi    /home/pi/solarmax/solarmax-mqtt-pub.sh

On the mqtt server, check that messages are arriving using mosquitto_sub:

$ mosquitto_sub -h <mqtt_server> -t solarmax -u <mqtt_user> -P <mqtt_password> -p 1883
[{"name": "3000S", "addr": "03", "pac": "3.0", "kwh_today": "13.4"}, {"name": "6000S", "addr": "06", "pac": "33.0", "kwh_today": "28.3"}]
[{"name": "3000S", "addr": "03", "pac": "6.0", "kwh_today": "13.4"}, {"name": "6000S", "addr": "06", "pac": "35.0", "kwh_today": "28.3"}]

After sunset, the inverter becomes unreachable because it turns off itself while the PV panels do not generate any energy.

Therefore we just stop sending mqtt messages until next dawn.

• Tags:
• grafana influxdb nodered mqtt mosquitto

• ← Monitor your FRITZ!Box Internet bandwidth usage using lots of free software: Python, Redis, Raspbian, Mosquitto, Node-RED, InfluxDB, Grafana
• Volvo XC60 cannot play (some) MP3 tracks →