[DEV] MycroftOS – A bare minimal OS based on Buildroot – PART 2

[DEV] MycroftOS – A bare minimal OS based on Buildroot – PART 2

In my previous first blog post about the MycroftOS project, I informed you all about the start of the project. Now I did mention in that blog post that it was most likely a bit premature to already blog about it. However in the last few weeks I have been able to work on the project and it slowly becomes at the point that people could download, compile and install the firmware onto their device to have a look. There are still some small bugs that I am working on. However the fundementals are in place. You could look around, setting up you microphone and such and play around a bit. Hopefully if you do so, report back the issues which you will definitely encounter.

Please be aware; We are NOT there yet !!!

So where are we at? This second blog post within the series is to give you all a bit more information of what has been done since the last blog post, what hasn’t been done yet and what I have planned for this firmware in the next few weeks. As always I am fully open for your ideas and feedback. Both positive as negative.

The code of the project is at my Github page. Commit / History is currently a HUGE mess as I am a fan of commit early and often, do the cosmetics at a later stage. Commiting often allows me to test small pieces and bits, ussually try and error stuff without getting lost in all small attempts of what I changed where etc. Secondly I like to push my work to Github as well. Pure for backup purposes.

So yeah, commits and history does look very messy and Yes you might get a good laugh at me at certain stuff, but I don’t really care. As soon as I figured out all the right stuff for a initial release I will rebase and squeeze it all in a few big commits anyway. OK, that rebase / re-factoring has happened. See for more information the next blog post

So with that out of the way, where are we at this moment. A few pointers of the current status;

  • Based on the last buildroot release : 2018.05.x.
  • Image filesystem size for now fixed at 1 GB.
  • Filesystem formatted as EXT2/4.
  • All required software packages and libraries installed at build time.
  • All required python packages installed at build time.
  • ALSA configured for 3.5mm jack as default
  • Pulseaudio as system wide deamon
  • Mycroft (and all python dependencies for that matter) installed system wide (no need for virtual env’s)
  • All Mycroft services are started at boot time.
  • No wifi configuration yet, LAN configured for DHCP
  • System boots in about 5 seconds to CLI
  • First boot, means first start of Mycroft which means all skills get installed in /opt/mycroft which takes some time, but in about 20 seconds you hear the register code over the speakers.

I probably miss some pointers here and there, but if so I will add them as soon as I remember I forgot anything.

Like a said earlier above, there are still some bugs and quircks that I need to sort before I can start with my nice-to-have list. The first one is actually about ALSA in combination of the Raspberry Pi not having a “Master” violume control by default. (Only PCM);

[bash]

VolumeSkill – ERROR – Couldn’t allocate mixer, ALSAAudioError(‘Unable to find mixer control Master,0 [default]’,)

[/bash]

I recently saw similar reports online about the last Stretch based Picroft image. It most likely can be solved by creating a SoftVol virtual Master device, such as;

[bash]

pcm.softvol {
type softvol
slave {
pcm "hw:0"
}
{
name "Master"
card 0
}
}

[/bash]

Regardless, I do believe this is a minor thing to fix, however needed to properly control the volume from Mycroft itself.

Next on the list is actually a “locale” setting bug. Two skills basically fail because of Unicode / ASCI decoding which in turn is most likely caused by the wrong or perhaps in our case missing locale “en_US.UTF-8” settings.

Below the two errors I managed to catch from the Skills log;

[bash]

Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/mycroft/skills/core.py", line 136, in load_skill
raise e
File "/usr/lib/python3.6/site-packages/mycroft/skills/core.py", line 129, in load_skill
skill.load_data_files(path)
File "/usr/lib/python3.6/site-packages/mycroft/skills/core.py", line 1022, in load_data_files
self.init_dialog(root_directory)
File "/usr/lib/python3.6/site-packages/mycroft/skills/core.py", line 1013, in init_dialog
self.dialog_renderer = DialogLoader().load(dialog_dir)
File "/usr/lib/python3.6/site-packages/mycroft/dialog/__init__.py", line 123, in load
join(path, f))
File "/usr/lib/python3.6/site-packages/mycroft/dialog/__init__.py", line 49, in load_template_file
for line in f:
File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xe2 in position 22: ordinal not in range(128)

Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/mycroft/skills/core.py", line 136, in load_skill
raise e
File "/usr/lib/python3.6/site-packages/mycroft/skills/core.py", line 129, in load_skill
skill.load_data_files(path)
File "/usr/lib/python3.6/site-packages/mycroft/skills/core.py", line 1022, in load_data_files
self.init_dialog(root_directory)
File "/usr/lib/python3.6/site-packages/mycroft/skills/core.py", line 1013, in init_dialog
self.dialog_renderer = DialogLoader().load(dialog_dir)
File "/usr/lib/python3.6/site-packages/mycroft/dialog/__init__.py", line 123, in load
join(path, f))
File "/usr/lib/python3.6/site-packages/mycroft/dialog/__init__.py", line 49, in load_template_file
for line in f:
File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xe2 in position 15: ordinal not in range(128)

[/bash]

The second error comes from the fallback-unknown.mycroftai skill. Forgot about the first one, however they most likely require the same fix. Fixing one will fixing both.

Then the last python based error which is caused by a skill is an error in feedparser.py

[bash]

Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/mycroft/skills/core.py", line 118, in load_skill
(‘.py’, ‘rb’, imp.PY_SOURCE)
File "/usr/lib/python3.6/imp.py", line 235, in load_module
return load_source(name, filename, file)
File "/usr/lib/python3.6/imp.py", line 172, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 684, in _load
File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 650, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/opt/mycroft/skills/mycroft-npr-news.mycroftai/__init__.py", line 15, in <module>
import feedparser
File "/usr/lib/python3.6/site-packages/feedparser.py", line 316
raise KeyError, "object doesn’t have key ‘category’"
^
SyntaxError: invalid syntax

[/bash]

Not 100% where this error comes from as a quick google session showed it is an error in feedparser based on the python 2 to 3 migration. However the version installed is way above the version that should have python 3.6 support. It might be that the missing locale is also causing this one, so will leave this one to after the locale. Perhaps it also solves this error.

However all services are started and everything is being populated as it should. Not all skills are already 100% error free, but those errors will be fixed shortly.

Last error that I saw is not by any of the services but more in starting the speech-client-cli debug screen.

[bash]

Traceback (most recent call last):
File "/usr/bin/mycroft-cli-client", line 11, in <module>
load_entry_point(‘mycroft-core==18.8.2’, ‘console_scripts’, ‘mycroft-cli-client’)()
File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 572, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2755, in load_entry_point
return ep.load()
File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2408, in load
return self.resolve()
File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2414, in resolve
module = __import__(self.module_name, fromlist=[‘__name__’], level=0)
File "/usr/lib/python3.6/site-packages/mycroft/client/text/__main__.py", line 60, in <module>
main()
File "/usr/lib/python3.6/site-packages/mycroft/client/text/__main__.py", line 55, in main
curses.wrapper(gui_main)
File "/usr/lib/python3.6/curses/__init__.py", line 94, in wrapper
return func(stdscr, *args, **kwds)
File "/usr/lib/python3.6/site-packages/mycroft/client/text/text_client.py", line 1071, in gui_main
c = scr.get_wch() # unicode char or int for special keys
AttributeError: ‘_curses.window’ object has no attribute ‘get_wch’

[/bash]

However again I have the missing locale as possible cause of this as well. So at the moment I am figuring out how to install locale settings and set them to en_US.UTF-8 as default. This most likely will fix the last small issues. From there on we should have a bare minimal installation of Mycroft on Buildroot with all the needed packages installed. However nothing more than that, so it remains very slim and fast. Exactly what we want.

Then onto the last section of this blog post. What I have planned to work on after fixing these last issues. Again wrapped up in a few bullit points;

  • Auto resize partition. Fixed at 1GB for now (can and need to be extended manually if needed), but in the end minimised for download and flashing. First boot will then auto extend the partition over the whole size of the SD card.
  • Implementying the same or similar setup wizards to configure Mycroft audio and such.
  • Wifi implementation, similar as what Google does. Connect your phone / mobile device to MycroftOS AP SSID, redirect to a small configuration webpage hosted on the MycroftOS device where you can setup the Wifi and perhaps later on in time, other settings as well.
  • ReSpeaker MIC array drivers as Buildroot package so that it will include those driver and loads them if it sees any of the supported hardware from Seeedstudio.

Above makes it a final kind of project to be released for the public, after that I will have a look at supporting the Mark-1 (and perhaps Mark-2), enclosures for ReSpeaker, and well basically all the nice to have ideas that I already ventilated in the previous blog post.

Keep you guys posted !


Do you like what you just read? Keep me going by donating on Paypal or become a Patron.