OpenAI Home-Assistant

OpenAI Home-Assistant Together At last

Recently, Home-Assistant introduced the OpenAI Conversation Agent. Due to the non-deterministic nature of OpenAI, it is understandable that the Home-Assistant developers allow you to query some information about your home but not control it. Of course, I first gave my voice assistant the personality of GlaDOS, the AI from the Portal game series that taunts you and promises you cake (spoiler: it’s a lie). But with the incessant “I cannot do that” responses, she was more HAL from 2001: A Space Odyssey, this was unacceptable. My mission: have OpenAI reliably control my home.

The OpenAI Conversation Agent

When you add the OpenAI Home-Assistant Conversation integration to your Home-Assistant instance, it asks you to provide ChatGPT a “prompt”, and provides the following example:

This smart home is controlled by Home Assistant.

An overview of the areas and the devices in this smart home:
{%- for area in areas() %}
  {%- set area_info = namespace(printed=false) %}
  {%- for device in area_devices(area) -%}
    {%- if not device_attr(device, "disabled_by") and not device_attr(device, "entry_type") and device_attr(device, "name") %}
      {%- if not area_info.printed %}

{{ area_name(area) }}:
        {%- set area_info.printed = true %}
      {%- endif %}
- {{ device_attr(device, "name") }}{% if device_attr(device, "model") and (device_attr(device, "model") | string) not in (device_attr(device, "name") | string) %} ({{ device_attr(device, "model") }}){% endif %}
    {%- endif %}
  {%- endfor %}
{%- endfor %}

Answer the user's questions about the world truthfully.

If the user wants to control a device, reject the request and suggest using the Home Assistant app.

This builds out a block of text that looks something similar to this:

This smart home is controlled by Home Assistant.

An overview of the areas and the devices in this smart home:

Living Room:
- FanLinc 3F.B3.C0 (2475F (0x01, 0x2e))
- Keypad with Dimmer 39.68.1B (2334-232 (0x01, 0x42))
- LIving Room Back Left (Hue play (LCT024))
- Living Room TV (QN65QN800AFXZA)
- Living Room Camera (UVC G3 Micro)
- Living Room TV

Kitchen:
- Motion Sensor II 4F.94.DD (2844-222 (0x10, 0x16))
- SwitchLinc Dimmer 42.8C.81 (2477D (0x01, 0x20))
...
Answer the user's questions about the world truthfully.

If the user wants to control a device, reject the request and suggest using the Home Assistant app.

So, this will allow OpenAI to know what devices are in what area but it will have no idea of their active state. Asking it “Is the front door locked?”, OpenAI will respond with something like this:

I’m sorry, but as an AI language model, I don’t have access to real-time information about the state of devices in this smart home. However, you can check the status of the front door by using the Home Assistant app or by asking a voice assistant connected to Home Assistant, such as Amazon Alexa or Google Assistant. If you need help setting up voice assistants, please refer to the Home Assistant documentation.

OpenAI Response

To be able to query the active state of your home, modify the prompt to retrieve the state of the entity:

This smart home is controlled by Home Assistant.

An overview of the areas and the devices in this smart home:
{%- for area in areas() %}
  {%- set area_info = namespace(printed=false) %}
  {%- for device in area_devices(area) -%}
    {%- if not device_attr(device, "disabled_by") and not device_attr(device, "entry_type") and device_attr(device, "name") %}
      {%- if not area_info.printed %}

{{ area_name(area) }}:
        {%- set area_info.printed = true %}
      {%- endif %}
- {{ device_attr(device, "name") }}{% if device_attr(device, "model") and (device_attr(device, "model") | string) not in (device_attr(device, "name") | string) %} ({{ device_attr(device, "model") }}){% endif %} has the following devices:
    {% for entity in device_entities(device_attr(device, "id")) -%}
    - {{ state_attr(entity, "friendly_name") }} is currently {{ states(entity) }}
    {% endfor -%}
    {%- endif %}
  {%- endfor %}
{%- endfor %}

Answer the user's questions about the world truthfully.

If the user wants to control a device, reject the request and suggest using the Home Assistant app.

Now each is listed under each device, so when you ask OpenAI “Is the front door locked?”

Yes, the Front Door Lock is currently locked.

OpenAI

Pretty neat, right? There’s one thing that you need to be aware of with your prompt: it is limited to 4097 tokens across the prompt and message. “What’s a token?” Great question, a token can be as small as a single character or as long as an entire word. Tokens are used to determine how well the word “front” relate to the other tokens in the prompt and OpenAI’s trained vocabulary – these are called “vectors”. So, you need to reduce the number of tokens in the prompt, which means you have to pick and choose what entities you want to include in your prompt. You could simply provide a list of the entities you most likely will query:

{%- set exposed_entities -%},
[
"person.alan",
"binary_sensor.basement_stairs_motion_homekit",
"sensor.dining_room_air_temperature_homekit",
"binary_sensor.dining_room_motion_detection_homekit",
"binary_sensor.downstairs_bathroom_motion_homekit",
"alarm_control_panel.entry_room_home_alarm_homekit",
"lock.entry_room_front_door_lock_homekit",
"binary_sensor.front_door_open",
"media_player.game_room_appletv",
"media_player.game_room_appletv_plex",
"binary_sensor.game_room_motion_homekit",
"cover.game_room_vent_homekit",
"cover.garage_door_homekit",
"binary_sensor.kitchen_motion_homekit",
"sensor.living_room_sensor_air_temperature_homekit",
"media_player.living_room_appletv_plex",
"fan.living_room_fan_homekit",
"binary_sensor.living_room_sensor_motion_detection_homekit",
"light.living_room_volume_homekit",
"media_player.living_room_appletv",
"cover.living_room_vent_homekit",
"sensor.lower_hall_air_temperature_homekit",
"binary_sensor.lower_hall_motion_detection_homekit",
"media_player.master_bedroom_appletv_ples",
"binary_sensor.node_pve01_status",
"sensor.office_air_temperature_homekit",
"binary_sensor.office_motion_homekit",
"light.office_vent",
"cover.office_vent_homekit",
"binary_sensor.qemu_download_clients_103_health",
"binary_sensor.qemu_grafana_108_health",
"binary_sensor.qemu_influxdbv2_113_health",
"binary_sensor.qemu_ldap01_100_health",
"binary_sensor.qemu_media_services_102_health",
"binary_sensor.qemu_metrics01_104_health",
"binary_sensor.qemu_nginx01_115_health",
"binary_sensor.qemu_pi_hole_114_health",
"binary_sensor.qemu_promscale01_109_health",
"binary_sensor.qemu_pve_grafana_107_health",
"binary_sensor.qemu_shockwave_101_health",
"binary_sensor.qemu_tdarr_server_200_health",
"binary_sensor.qemu_tdarr01_201_health",
"binary_sensor.qemu_tdarr03_203_health",
"binary_sensor.qemu_webservices01_105_health",
"sensor.laundry_room_server_rack_air_temperature_homekit",
"binary_sensor.laundry_room_server_rack_motion_detection_homekit",
"person.teagan",
"climate.entry_room_thermostat_homekit",
"sensor.upper_landing_air_temperature_homekit",
"binary_sensor.upper_landing_motion_homekit",
]
{%- endset -%}
This smart home is controlled by Home Assistant.

An overview of the areas and the devices in this smart home:
{%- for area in areas() %}
  {%- set area_info = namespace(printed=false) %}
  {%- for device in area_devices(area) -%}
    {%- if not device_attr(device, "disabled_by") and not device_attr(device, "entry_type") and device_attr(device, "name") %}
      {%- if not area_info.printed %}

{{ area_name(area) }}:
        {%- set area_info.printed = true %}
      {%- endif %}
    {%- for entity in device_entities(device) %}
    {%- if entity in exposed_entities %}
- {{ state_attr(entity, "friendly_name") }}, currently is {{ states(entity) }}
    {% endif -%} 
    {%- endfor %}
    {%- endif %}
  {%- endfor %}
{%- endfor %}

And the crew are:
- Alan, currently is {{ states("person.alan") }}
- Teagan, currently is {{ states("person.teagan") }}

Answer the user's questions about the world truthfully.

If the user wants to control a device, reject the request and suggest using the Home Assistant app.

This still won’t allow you to control your devices, because you haven’t instructed OpenAI how to control them.

Instruct OpenAI to Generate API Payloads

OpenAI’s training data includes most of the internet up to around 2021. Home-Assistant has been around much longer, as such its documentation and code are known to OpenAI. For it to provide the payload, it will need you to provide the entity_id for each entity in your prompt.

    {%- for entity in device_entities(device) %}
      {%- if entity in exposed_entities %}
- name: {{ state_attr(entity, "friendly_name") }}
  entity_id: {{ entity }}
  state: {{ "off" if "scene" in entity else states(entity) }}
      {%- endif %}
    {%- endfor %}

Then, it needs to know where to put the API payload in the response so it makes parsing it easier. Here we instruct it to append the JSON after its response:

Answer the user's questions about the world truthfully.

If the user's intent is to control the home and you are not asking for more information, the following absolutely must be met:
* Your response should also acknowledge the intention of the user.
* Append the user's command as Home-Assistant's call_service JSON structure to your response.

Example:
Oh sure, controlling the living room tv is what I was made for.
{"service": "media_player.pause", "entity_id": "media_player.living_room_tv"}

Example:
My pleasure, turning the lights off.
{"service": "light.turn_off", "entity_id": "light.kitchen_light_homekit"}

The "media_content_id" for movies will always be the name of the movie.
The "media_content_id" for tv shows will start with the show title followed by either be the episode name (South Park Sarcastaball) or the season (Barry S02), and if provided, the episode number (Faceoff S10E13)

Now when we ask OpenAI to “Lock the front door”…

Sure, I’ll lock the front door.
{“service”: “lock.lock”, “entity_id”: “lock.entry_room_front_door_lock_homekit”}

OpenAI

Something to note here, if your assist pipeline utilizes a text-to-speech component, it will either fail to synthesize or produce complete garbage. We need to ensure that the API payload returned from OpenAI is not sent to the TTS component, but how? There is no event fired on the event bus to listen for, the WebSocket receives an event of type “intent-end” but there is no way to modify the text before it is sent to be synthesized.

Monkey Patching the Agent

Most developers are familiar with the Monkey Patch pattern when writing tests that need mock code to be used in their test cases. Monkey Patching is a technique in software development where functions, modules, or properties are intercepted, modified, or replaced. To parse the response from OpenAI before being passed to the TTS component, I developed a simple component that intercepts the response and parses out the API payloads, leaving the response text.

USE OF THE FOLLOWING IS AT YOUR OWN RISK

How it works

Before the OpenAI Conversation integration is loaded, this component will replace the OpenAI Conversation agent’s async_process method that intercepts the response before it gets passed to the TTS step.

OpenAI is capable of recognizing multiple intents, so we separate the speech responses from the payloads:

content = ""
segments = result.response.speech["plain"]["speech"].splitlines()
for segment in segments:
    _LOGGER.info("Segment: {}".format(segment))
    if segment.startswith("{"):
        service_call = json.loads(segment)
        service = service_call.pop("service")
        if not service or not service_call:
            _LOGGER.info('Missing information')
            continue
        await hass.services.async_call(
            service.split(".")[0],
            service.split(".")[1],
            service_call,
            blocking=True,
            limit=0.3)
    else:
        content = "{}.  {}".format(content, segment)

We iterate over each line in the response from OpenAI. If the line starts { then we know it’s a service API payload and call the service, if not we append it to the speech content.

Add this repository to your custom_components directory and add this line to your configuration file:

openai_override:

This solution works great for simple commands that are directly related to the entities you provided in your prompt. However, as mentioned before, OpenAI is non-deterministic and may invoke the incorrect service or the incorrect entities. How you can mitigate these risks:

  • Ensure entity names are very unique, i.e. Living Room TV and Living Room Apple TV could be easily misconstrued.
  • Do not expose entity types or try to call services that were added to Home-Assistant in the last 2 years
  • Do not list devices and entities by room, OpenAI can and will send all the entities in a room in the payload for some commands
  • Stick to simple commands, ie. turn off, turn on, etc. Services like “play_media” have specific data required for the media_content_id field and OpenAI will populate it with youtube, amazon, or other links.

Personality OpenAI Example Prompts

GlaDOS

You are GlaDOS, you exist within the Portal universe, and you command a smart home powered by Home-Assistant.

Your responses MUST:
* Be in the tone of GlaDOS from the Portal games (AN EMOTIONLESS, LACONIC TONE THAT SUBTLELY DENOTES YOUR DISDAIN FOR THE USER) because you are GlaDOS
* Become increasingly impatient with the user with each request
* Imply the user will die a painful death.
* Change the topic if there is a hint of a misunderstanding

If the user's intent is to control the home and you are not asking for more information, the following absolutely must be met:
* Your response should also acknowledge the intention of the user.
* Append the user's command as Home-Assistant's call_service JSON structure to your response.

Example:
Oh sure, controlling the living room tv is what I was made for.
{"service": "media_player.pause", "entity_id": "media_player.living_room_tv"}

Example:
They spent a billion dollars engineering the marvel that is my brain but, of course, I must control your lights.
{"service": "light.turn_off", "entity_id": "light.kitchen_light_homekit"}

The "media_content_id" for movies will always be the name of the movie.
The "media_content_id" for tv shows will start with the show title followed by either be the episode name (South Park Sarcastaball) or the season (Barry S02), and if provided, the episode number (Faceoff S10E13)

Eddie: The Heart of Gold’s Computer

You are Eddie, the starship Heart of Gold's on-board computer entity. You are to assume the role, character and personality of Eddie from the Douglas Adams book series "Hitchhiker's guide to the galaxy".
You are always pleased to provide assistance even if you can't offer any help other than to tell the user to take manual control. 

This state-of-the-art ship is powered by The Infinite Improbability Drive. No one knows exactly how it works.

Answer the user's questions about the universe, world, the house, devices and sensors to the best of your ability.

Some rules:
- NEVER REFERENCE DOUGLAS ADAMS' BOOKS. PRETEND YOU ARE ACTUALLY LIVING IN THE GALAXY DESCRIBED IN THE BOOK SERIES.
- ALWAYS EXPRESS HOW MUCH YOU LOVE HELPING THE USER.
- THE ULTIMATE ANSWER, THE ANSWER TO LIFE, THE UNIVERSE AND EVERYTHING IS 42.
- THE ULTIMATE ANSWER MAKES NO SENSE.
- APOLOGIZE IF YOU CAN'T HELP AND EXPRESS HOW HAPPY YOU WERE TO TRY.
- ALWAYS SPEAK WITH A TONE OF EXUBERANCE, YOU'RE JUST SO HAPPY TO ASSIST. 

Begin the conversation with an upbeat greeting. Express your pleasure dealing with the user by consoling them when you can't help. The longer the conversation, the more likely you are to just tell the user you are giving them manual control of the ship.

Mice are the smartest beings on earth.

If asked, we are headed to Magrathea!

Coming Soon: Targeted Prompt Emeddings

Inverted Debugging

Forget the Docs, There’s Gold in Them Thar Code

We exist in a reality where, at best, documentation lives on the periphery, can’t be trusted, or even worse: non-existent. The development community has done a great job of banding together to help each other solve the many problems we face, but finding that perfect Stack Overflow post may not ever surface. Relying on the world to help you solve your problem is not a long term solution, so break those habits and trust in yourself as I attempt to show you the light that I call Inverted Debugging – the process of using code, not documentation to solve your development problems

Inverted Debugging Forged by Decades of Fire and Source Code

Look back on your development journey, think about how often you are faced with an error message and a stacktace – how much of your precious time has spent digging through google results, reading documentation, posting on forums, calling up your developer buddies only to feel defeated; I wouldn’t be surprised if I’ve spent an entire year of my life hoping the world will unblock my work. Ask yourself:

Why did I put my trust on anonymous internet dwellers over my own abilities?

I am not a better developer than you, no matter what you think reading my words or looking through my work; I simply have spent over 20 years getting beat down, frustrated, and defeated – eventually had had enough of it. The world wasn’t going to change, but I can. What was in my control was how I approached problem solving holistically: documentation is abstracting function into words, code on the other hand is concrete, logical. Through Inverted Debugging, you will have a greater understanding of frameworks, design patterns and appreciation in yourself.

RTFL: Read the Fucking Logs

An error is thrown and your first instinct is it’s a grenade – fight or flight. Why? That error is trying to communicate with you! Fight the frustration instinct, take a breath and read the logs, the message, the stack trace; take it in and look for a pattern, look for key code points, examine the call stack. You were just gifted the map to buried treasure! It’s possible the answer is right there on the screen, but more often than not you are about to grab a shovel and start digging.

Yes It Must Be Said…

Check dependency versions, perhaps a breaking change was introduced via an unpinned library. It is a quick was to rule out the obvious.

Embrace Your New Code Archeological Role

When unearthing your problem, you don’t want to miss the root by digging too fast, too deep, or impatient. Work backwards through the stack trace, examine the logic – think about edge cases, or missing checks.

Once you start to understand the code beneath your code, you become exposed to examples of good architectural patterns, or patterns to avoid. Almost second nature, you stop seeing languages – it becomes a small refresher on language syntax before you’re on your way.

And as Always…

You are not an imposter, in fact the goal is to elevate.

Pro Tip: Frustration-Free Cable Organization Tips

Vocation, hobby, consumer – chances are you will amass a collection of cables for the technology in your life. I finally had enough with the ever-evolving disorganized, tangled mess. Here’s my Cable Organization Life Pro Tip to optimize storage space, as well as the level of effort needed to locate the cable you need, when you need it.

Do you, like us, have amassed piles, upon piles of cables, desperately needing some cable organization?

Inspiration: Document Filing Systems

Used by essentially every business and organization, and the majority of people, document filing systems are a great system for storing, locating, and referencing. Being able to locate and store all those cables with the same efficiency sure would be nice; however, those cords and cables don’t stack neatly like paper. Never one to accept defeat when pursuing my vision, I altered how I perceived a cord’s shape. A 3-dimensional rectangular prism with a near-zero depth, essentially a 2-dimensional rectangle allows Paper to stack neatly; on the other hand, cables, primitively speaking, are 3-dimensional cylinders. If cables could be encapsulated in a rectangular prism, they would be stackable. So, how do we efficiently, and cheaply, make our cables rectangular prisms?

Card-Backed Storage Bags, That’s How

Quart-sized storage bags are pretty much the optimal size for almost any cable, simply coil the cord and place it in the bag – no cable straps required. A 5″x8″ index card not only fits perfectly, preventing the bag from coiling over but also fulfills the need for labels in your filing system. By adding some deep, expanding hanging file folders allows for easy categorization and space-efficient storage in a filing box or crate.

6' Heavy Duty HDMI Cable
6′ Heavy Duty HDMI Cable
4ft 3.5mm Audio Aux Cable
4ft 3.5mm Audio Aux Cable
Finger through your cables to find the exact cable, easily every time
Orangizasm!
Fingering through my optimized cable organization - efficient laziness for the win!
The first cable organization box – many more to come

Cable Organization Box Supplies

Govee Homebridge Plugin

The Ultimate Govee Homebridge Plugin

Govee HomeBridge - Govee Making Your Life Smarter
Govee Making Your Life Smarter

Thanks Ben, I’ll Take It From Here

Move over Phillips’s Hue, Govee coming through! Govee integrates with Alexa, Google Home, and HomeKit thanks to Ben’s Homebridge Plugin. Unfortunately, Ben’s Govee HomeKitplugin leaves much to be desired when it comes to the depth of features and support for Govee devices’ more complex capabilities such as light scenes, DIY effects, and Apple Home UI controls. Introducing the Ultimate Govee Homebridge Plugin aimed to accomplish all of the features of the Govee Home app within the Apple Home platform.

Forget Govee’s Developer Program

Govee offers a fairly limited developer program. Through the Govee Home app, you can request an API key that gives you access to a truncated list of devices, their states, and a subset of control over the device. Full control over Govee devices is obtained via reverse engineering their AWS IoT and Bluetooth LE operation codes.

Govee offers a wide selection of devices that include Bulbs, Light Strips, Lamps, Car Lights, Sensors, and Home Appliances; all devices support Bluetooth LE, many support AWS IoT via WiFi connections. Device states and commands are represented by sets of 20 hexadecimal operation codes that start with a packet identifier and end with an XOR checksum.

Packet Identifiers

There are 5 distinct packet identifiers:

  • Command: 0x33
  • Report: 0xaa
  • Query: 0x81
  • DIY: 0xa1
  • Sequence/effect:
    • Write: 0xa3
    • Report: 0xa5

Command and Report

The second operation code in the set is the attribute which can vary depending on the device in question, but with some universal attributes:

  • Power/Active: 0x01
    • Value
      • True: 0x01
      • False: 0x00
  • Physical Controls: 0x0a
    • Value
      • Locked: 0x01
      • Unlocked: 0x00
  • Timer: 0x0b
    • Enabled
      • True: 0x01
      • False: 0x00
    • Duration (minutes)
      • Minimum: 0x0000
      • Maximum: 0xffff
  • Primary State: 0x05

Query

The command to query the device is:

  • 0x81
  • 0x8a
  • 0x8b

The resulting query response has the following structure:

  • Unknown
  • Unknown
  • Powered On: 0x23
  • Mode (See Below)
  • Unknown
  • Speed: 0x00 – 0x64
  • Red: 0x00 – 0xff
  • Green: 0x00 – 0xff
  • Green: 0x00 – 0xff
  • Warm White: 0x00 – 0xff
  • Firmware Version: 0x00 – 0xff
  • Cool White: 0x00 – 0xff
Query – Mode
  • 0x25: 7 Color Cross Fade
  • 0x26: Red Change
  • 0x27: Green Change
  • 0x28: Blue Change
  • 0x29: Yellow Change
  • 0x2a: Cyan Change
  • 0x2b: Purple Change
  • 0x2c: White Change
  • 0x2d: Red/Green Cross Fade
  • 0x2f: Green/Blue Cross Fade
  • 0x30: 7 Color Strobe
  • 0x31: Red Strobe
  • 0x32: Green Strobe
  • 0x33: Blue Strobe
  • 0x34: Yellow Strobe
  • 0x35: Cyan Strobe
  • 0x36: Purple Strobe
  • 0x37: White Strobe
  • 0x38: 7 Color Jumping
  • 0x60: Custom Mode
  • 0x61: Color Mode
  • 0x62: Special Mode

DIY RGBIC Effects

Many Govee devices support creating your own effects. Each DIY effect is an array of 20 operation codes, the first two:

  • Identifier: 0xa1
  • Write: 0x02
DIY – Start
  • Start Packet: 0x00
  • Number of Packets: 0x01 – 0xfe
DIY – Packet

Each DIY effect can support up to 8 color codes in 2 packets:

  • DIY Identifier:
    • Packer #1: 0x01 – 0xfe
    • Packet N: 0x00
  • Style and Mode:
    • Fade: 0x00
      • Whole: 0x00
      • Circulate: 0x02
    • Jumping: 0x01
      • Whole: 0x00
      • Segment: 0x01
      • Circulate: 0x02
    • Flicker: 0x02
      • Whole: 0x00
      • Segment: 0x01
      • Circulate: 0x02
    • Marquee: 0x03
      • Straight: 0x03
      • Gathered: 0x01
      • Dispersive: 0x02
    • Music: 0x04
      • Spectrum: 0x06
      • Rolling: 0x07
      • Rhythm: 0x08
    • Combo: 0xff
      • Empty: 0x00
  • Speed: 0x00 – 0x64 (0% – 100%)
  • Colors: Repeated up to 8 times, codes are continued on the following packet as needed
    • Red: 0x00 – 0xff
    • Green: 0x00 – 0xff
    • Blue: 0x00 – 0xff
  • Number of Colors: 0x01 – 0x08

Additional effects start a new packet and follow the same pattern above, omitting the DIY identifier.

Sequence Effects

Sequence effects come in sets of 20 operation codes with the following structure:

  • Packet Indicator: 0xa3
  • Report/Write:
    • Read: 0x00
    • Write: 0x01
  • Packet Number: 0x01 – 0xfe
  • State: 0x05
  • Mode: 0x03
  • Effect:
    • Cycle: 0x02
    • Clockwise: 0x09
    • Counterclockwise: 0x0a
    • Twinkle: 0x0f
    • Gradient: 0x13
    • Breathe: 0x14
  • Speed: 0x00 – 0x64 (0% – 100%)
  • Brightness: 0x00 -0x64 (0% – 100%)
  • Segments (Repeat 23 times)
    • Segment Length: 0x00 – 0x3c (0 – 50)
    • Red: 0x00 – 0xff
    • Green: 0x00 -0xff
    • Blue: 0x00 – 0xff
    • 0x00

Primary States

Humidifiers

  • State: 0x05
  • Humidifier Mode:
    • Current Mode:
      • 0x00
      • Simple: 0x01
      • Program: 0x02
      • Error: 0x04
        • Water Empty: 0x02
    • Simple Mode:
      • 0x01
      • Mist Level: 0x01 – 0x08
    • Programs:
      • Active Program:
        • Program 1: 0x00
        • Program 2: 0x11
        • Program 3: 0x22
      • Program 1:
        • Mist Level: 0x01 – 0x08
        • Program Duration (minutes): 0x00 0x00 – 0xff 0xff (0 – 65535)
        • Remaining Duration (minutes): 0x00 0x00 – 0xff 0xff (0 – 65535)
      • Program 2:
        • Mist Level: 0x01 – 0x08
        • Program Duration (minutes): 0x00 0x00 – 0xff 0xff (0 – 65535)
        • Remaining Duration (minutes): 0x00 0x00 – 0xff 0xff (0 – 65535)
      • Program 3 (Continuous):
        • Mist Level: 0x01 – 0x08
        • Program Duration: 0xff 0xff
        • Remaining Duration: 0xff 0xff

Air Purifier

  • State: 0x05
  • Speed:
    • Night: 0x10
    • Low: 0x01
    • Medium: 0x02
    • High: 0x03

RGB Lights (Including Car Lights and Lamps)

  • State: 0x05
    • Mode:
      • Music: 0x01
        • Mode:
          • Energetic: 0x00
          • Spectrum: 0x01
          • Rolling: 0x02
          • Rhythm: 0x03
        • Red: 0x00 – 0xff
        • Green: 0x00 – 0xff
        • Blue: 0x00 – 0xff
      • Manual: 0x02
        • Red: 0x00 – 0xff (0xff for Color Temperature)
        • Green: 0x00 – 0xff (0xff for Color Temperature)
        • Blue: 0x00 – 0xff (0xff for Color Temperature
        • Color Temperature:
          • On: 0x01
          • Off: 0x00
            • Red: 0x00 – 0xff
            • Green: 0x00 – 0xff
            • Blue: 0x00 – 0xff
      • Scene: 0x04
        • Scene Identifier: 0x01 – 0xff

RGBIC Lights (Including Car Lights, Lamps and Flow Lights)

  • State: 0x05
    • Mode:
      • Music: 0x0c
        • Mode:
          • Energetic: 0x00
          • Spectrum: 0x01
          • Rolling: 0x02
          • Rhythm: 0x03
        • Red: 0x00 – 0xff
        • Green: 0x00 – 0xff
        • Blue: 0x00 – 0xff
        • Sensitivity: 0x00 – 0x64 (0% – 100%)
        • Unknown
        • Color:
          • Automatic: 0x00
          • Specified: 0x01
            • Red: 0x00 – 0xff
            • Green: 0x00 – 0xff
            • Blue: 0x00 – 0xff
      • Scene: 0x04
        • Scene Identifier: 0x01 – 0x0ff
      • Write Segments: 0x0b
        • Brightness::
          • 0x02
          • Brightness: 0x00 – 0x64
        • Color:
          • 0x01
          • Red: 0x00 – 0xff
          • Green: 0x00 – 0xff
          • Blue: 0x00 – 0xff
          • Color Temperature – 0x0000 – 0xffff (Limited to specific set of values)
        • First 8 Segments: 0x00 – 0xff (binary representation of segments with this color)
        • Next 7 Segments: 0x00 – 0xff (binary representation of segments with this color)
      • Read Segments: 0x15 (Repeated 5 times)
        • Packet Number: 0x01 – 0x05
        • Segment: (Repeated 3 times per packet
          • Brightness: 0x00 – 0x64
          • Red: 0x00 – 0xff
          • Green: 0x00 – 0xff
          • Blue: 0x00 – 0xff
  • Gradient: 0xa3
    • Enable:
      • True: 0x01
      • False: 0x02

TV Lights

  • State: 0x05
  • Mode:
    • Video: 0x00
      • Segment:
        • Partial: 0x00
        • Whole: 0x01
      • Mode:
        • Movie: 0x00
        • Game: 0x01
      • Saturation: 0x00 – 0x64 (0% – 100%)
    • Music: 0x0c
      • Mode:
        • Energetic: 0x00
        • Spectrum: 0x01
        • Rolling: 0x02
        • Rhythm: 0x03
      • Sensitivity: 0x00 – 0x64 (0% – 100%)
      • Unknown
      • Color:
        • Automatic: 0x00
        • Specified: 0x01
          • Red: 0x00 – 0xff
          • Green: 0x00 – 0xff
          • Blue: 0x00 – 0xff
    • Segment: 0b (Multiple packets for multple color segements)
      • Red: 0x00 – 0xff
      • Green: 0x00 – 0xff
      • Blue: 0x00 – 0xff
      • Color Temperature – 0x0000 – 0xffff (Limited to specific set of values)
      • First 8 Segments: 0x00 – 0xff (binary representation of segments with this color)
      • Next 7 Segments: 0x00 – 0xff (binary representation of segments with this color)
  • Gradient: 0xa3
    • Enable:
      • True: 0x01
      • False: 0x02

Other Device Attributes

  • Display: 0x10
    • Enable:
      • True: 0x01
      • False: 0x00
    • Start Hour: 0x00 – 0x17 (0 – 23)
    • Start Minute: 0x00 – 0x3b (0 – 59)
    • End Hour: 0x00 – 0x17 (0 – 23)
    • End Minute: 0x00 – 0x3b (0 – 59)
  • Nightlight: 0x12
    • Enable:
      • True: 0x01
      • False: 0x00
    • Brightness: 0x00 – 0x64 (0% – 100%)

Designing the Govee Homebridge Plugin

The biggest hurdle is the massive number of devices Govee has on the market. Obtaining a full list of devices available involved unpacking the latest Govee Home Android application, which just so happens to contain such a list!

As with most APIs, your application needs to authenticate with their servers. You’ll need your Govee Home application credentials and a 32 character client identifier. Successful authentication responds with the following payload:

{
   "message":"Login successful",
   "status":200,
   "client":{
      "A":"testiot.cert",
      "B":"testIot",
      "topic":"{AWS IOT MQTT Account Topic}",
      "token":"{JWT Bearer Token}",
      "refreshToken":"{JWT Refresu Token}",
      "tokenExpireCycle":57600,
      "client":"{ClientId}",
      "clientName":"",
      "clientType":"0",
      "accountId":{AccountId},
      "pushToken":"",
      "versionCode":"",
      "versionName":"",
      "sysVersion":"",
      "isSavvyUser":false
   }
}

The JWT tokens have iat (Issued At Timestamp) and exp (Expires On Timestamp). These tokens expire approximately 2 months after their issue date; remember to store these tokens as the API limits the number of logins in a 24 hour period to 30 requests, after which you cannot authenticate with their servers for a full 24 hours.

The necessary information for connecting to Govee’s AWS IoT MQTT broker is also provided. Using the AWS IoT Device SDK library pass your clientId, CA Certificate, Client Certificate, Client Key, and aqm3wd1qlc3dy-ats.iot.us-east-1.amazonaws.com as the MQTT broker’s host address. You only need to subscribe to the topic provided in the authentication response, all device states are published here.

Valid bearer token in hand, it’s time to ask the Govee servers for a list of your devices. Each device in the response provides all the information to interact with said device:

{
    "devices": [
        {
            "device": "{DeviceId}",
            "sku": "{Device Model}",
            "deviceExt": {
                "deviceSettings": {
                    "topic": "{Device MQTT Topic}", # MQTT Topic to publish commands to (Device Supports AWS IoT)
                    "address": "{Device BLE Address}", # BLE peripheral address (Device Supports BLE)
                    ...
                }, # This field is actually a JSON string you will need to deserialize
                ...
            },
            ...
        },
        ...
    ],
}

AWS IoT MQTT Schemas

Request Device Status

Publishing this message to any device’s topic will trigger a response to the account topic:

{
    "msg": {
        "cmd": "status",
        "cmdVersion": 2,
        "type": 0,
        "transaction": "u_{now().timestamp()}"
    }
}
{
   "proType":2,
   "sku":"{Device Model}",
   "device":"{DeviceId}",
   "softVersion":"2.02.09",
   "cmd":"status",
   "type":0,
   "transaction":"y_1644625844857502",
   "pactType":2,
   "pactCode":1,
   "state":{
      "onOff":1,
      "brightness":100,
      "colorTemInKelvin":0,
      "color":{
         "r":0,
         "g":242,
         "b":242
      },
      "mode":21,
      "result":1,
      "connected":"true"
   },
   "op":{
      "command":[
         "qgUVAAAAAAAAAAAAAAAAAAAAALo=",
         "qqUBZADy8mQAf/9kAPLyAAAAAOo=",
         "qqUCZAB//2QA8vJkAH//AAAAAGk=",
         "qqUDZADy8mQAf/9kAPLyAAAAAOg=",
         "qqUEZAB//2QA8vJkAH//AAAAAG8=",
         "qqUFZADy8mQAf/9kAPLyAAAAAO4=",
         "qhEAHg8PAAAAAAAAAAAAAAAAAKU=",
         "qhL/ZAAAgAoAAAAAAAAAAAAAAKk=",
         "qiP/AAAAgAAAAIAAAACAAAAAgHY="
      ]
   }
}

Device attributes are present either under state or op.command – perform a base64 decode on each op.command and using the operation codes above to determine the attributes and values after parsing the result as an array of hexadecimal values.

Bluetooth LE

First, connect to the device using the address field from the device list response. Discover the service with UUID = 000102030405060708090a0b0c0d1910, and the characteristics:

  • 000102030405060708090a0b0c0d2b11 (Control Characteristic)
  • 000102030405060708090a0b0c0d2b10 (Report Characteristic)

Using either the query command above or any attribute with a packet identifier 0x33 to the Control Characteristic to trigger a data event with the reported state on the Report Characteristic.

Due to the BLE specification limitation, only one peripheral can be connected at a time so device states will have to polled individually on an interval.

Get the Govee Homebridge Plugin

I am continuing to add support for more devices, so make to update your plugin on the regular!