dt.in.th

automatron

I recommend every developer try building a personal assistant chat bot. It’s a great way to improve skill and make life easier at the same time.

I built automatron to be my personal LINE (opens new window) bot (opens new window) that helps me automate various tasks of everyday life:

  • home control (air conditioner, lights and plugs)
  • expense tracking (record how much I spend each day)
  • notetaking
  • quick calculations
  • etc.

Photo of my phone with chat logs

There’s a popular saying: “There’s an app for that.” Yes, but that means if I want to do 5 things, I had to switch between 5 apps. By building my own bot, I can integrate all these features deeply.

The source code is published on GitHub:

GitHub repository https://github.com/dtinth/automatron

# The story

automatron is the result of my laziness.

# I want my room to be cool already when I get there

Thailand is a hot country. I’ve always wanted to build was a system that let me remotely turn on the air-condition in my room while I travel.

I have no hardware hacking experience, so this project remained only an idea, until one day I learned that programmable IR transceivers exist. Oh, and Google Assistant is now a thing.

So I bought a Broadlink RM Mini3 and hooked it up to my Google Assistant (opens new window), along with other smart home stuff, like Philips Hue (opens new window) and TP-Link Kasa (opens new window).

This works fine but there are a few annoyances…

  • Google Assistant would (slowly) run the actions one-by-one.
  • In total it would take ~10 seconds to finish the routine.
  • If I switch to other apps while the routine is running, it will get interrupted.

After I implemented home automation functionalities into automatron, now I just send it stickers. It controls all the devices at once.

home automation

# I want to track my expenses

2014, I’m in my last year of college. I start taking internships and making some money. I feel the need to keep track of my expenses.

At first, I used an Apple Numbers spreadsheet. While this works, it takes a lot of effort to track of everything. I would need to keep my receipt (or take a photo of it) and update the sheet at the end of the day. I could also do it on my phone as the day goes by, but iCloud was a lot of hassle — occasionally it would say “your files are modified on both devices, choose one to keep.”

“Let’s build a dedicated app for this!” I thought. Then I built a web application to help me track my expenses. Still too many clicks. Gave up. I even made a voice interface. Still quite a hassle to use.

Then I realized… As a lazy programmer, I like to write more than being asked. Instead of filling in forms, I would rather say something short and unambiguous. If I had a personal chatbot, I can invent my own language that would concisely represent an expense. For example, 35f could mean ฿35 for food.

expense tracking

And here it is. No more filling forms. Data is saved in Airtable (opens new window).

Why I use Airtable

For expense tracking, ~95% of the time, the use case is to insert a new expense entry. I rarely edit/delete existing entries, so I don't want to spend time implementing it myself. But they still do happen.

I chose Airtable because then I can deep-link the reply into the Airtable app, and use it for edit/delete.

# I want to see all the transactions in one place

I have multiple bank accounts and each bank has its own app. But since I use an Android phone, I can parse incoming SMS from multiple banks, aggregate them in one place, and record expenses automatically.

transaction_aggregation

I set up IFTTT to read SMS messages (opens new window) and send it to automatron. It then uses transaction-parser-th (opens new window) to parse SMS messages and extract transaction information.

# Why not offer the option to track the transaction as expense as well?

I start to see a pattern… Most of the transactions are expenses, and I want to track them. Now I want to streamline this process, so I added some quick reply buttons (opens new window) to the transaction message.

Now when I receive a transaction, I can just tap on the corresponding category to track it as an expense.

quick_replies

# Why not track some expenses automatically?

While most transactions can’t be categorized automatically when all it knows is “paid ฿55 to 7-ELEVEN”. But some can. If I paid “BTS SkyTrain” then obviously it’s for transportation.

auto_expense

# I need to do some quick calculations

As a handy feature, I included a LiveScript (opens new window) interpreter. This allows me to do some quick calculations.

livescript

# Scheduled commands

Because automatron uses a text-based interface, implementing scheduled commands becomes very easy. Now I can tell my bot before sleeping “in 6h lights on”, and 6 hours later it would turn my lights back on.

Airtable table showing scheduled commands

With this I can also implement a command macros. For example, gn (goodnight) means lights dimmed and then in 5m lights off. (When I say goodnight to automatron, it would dim my room’s lights and turn it of 5 minutes later.)

The scheduled commands are also stored in Airtable, so if I need to cancel any scheduled command I can just edit it there.

# Image-to-text

As an extra feature, automatron can also convert images to text using Google Cloud Vision API (opens new window).

image_to_text

# History

published by at , last updated at