Welcome to my personal blog. If you're looking for my academic profile page, click the UCD in the upper right corner. Or maybe you're looking for my open source development blog?

Figuring out Rails API calls via WebExtensions XHR

16 May 2017

One of the questions I had in my last post was about how to manage sessions and login to a Rails app from a WebExtensions client. For example, let’s say your Rails app provides a Profiles resource that you handle in Rails with a route GET /profiles/:id. Let’s say you provide an API endpoint as well, at GET /api/v1/profiles/:id. From your WebExtensions script, you would issue the XHR like so:

var url = "http://localhost:4000/api/v1/profiles/2";
var req = new XMLHttpRequest();
req.open('GET', url, false);

Most likely, the endpoint will be protected by some sort of authentication check. The nifty thing is if you happen to have cookies from, say, having logged in through the web interface of the app, those are automatically sent when you send the XHR, and you can potentially be authenticated that way. But more typically for an API, you might be using Rails token authentication. Rails provides authenticate_or_request_with_http_token, which will look at the Authorization request header for a token that will be used instead of cookies. So you’ll need to do two things:

  1. get the token, and
  2. set it in the request header.

Conventionally, you might see the token made available by the app through some sort of user settings page where the user can copy and paste it into the extension to use for authorization. There might be a more automated way to do this, though.

The second part can be achieved by adding the following to the above snippet:

  var url = "http://localhost:4000/api/v1/profiles/2";
  var req = new XMLHttpRequest();
  req.open('GET', url, false);
+ req.setRequestHeader('AUTHORIZATION', 'Token token=your_token_here');

With this, Rails should be able to pick out the token from the request header and use that to authenticate the request and get your JSON response.

What’s next?

So while you now theoretically have full access to the API, there’s another thing to consider called Cross Origin Resource Sharing (CORS). Basically, it’s a way for a server to specify access control to resources from cross-site scripts, which is what our browser extension is. For example, we don’t want a malicious script to be able to send DELETE requests or otherwise mess with your data. CORS is a way to permit or deny requests, based on where the originating script is coming from.

This may not be an issue because a malicious script would somehow have to get access to the authentication token. But this may mean extra server-side setup depending on how this protocol is enforced across browsers. I’ll investigate the details in the next few days.

Rough plan for Perakun port

13 May 2017

What is Perakun

Perakun is a Firefox extension that helps Japanese learners by providing dictionary lookups for mouseover text. A great feature is its ability to save lookups to a word list. This gif gives a good summary:

State of Perakun

As I mentioned previously, Chrome has taken the lion’s share of the browser market, and Firefox has plans to move to an addon subsystem based on WebExtensions, which will be compatible with Chrome.

Perakun exists as a legacy addon in Firefox for Japanese and Chinese, but only the Chinese WebExtensions addon exists in Chrome, with only a subset of the functionality.

In theory, an addon written in WebExtensions should plug and play in any supporting browser, but there is enough inconsistency in API support where the code will need some updating.

Rough plan of action

  1. Update the Chinese Perakun WebExtensions addon to be fully cross-browser compatible.
  2. Update the addon to support Japanese dictionary.
  3. Restore word list functionality missing from legacy addon.
  4. Add functionality to post to HiNative server for richer learning experience.


Currently, I’ve gotten started fixing up the WebExtensions addon, which you can find on github. I’ve made some simple replacements to some deprecated calls, but it still isn’t functional in Firefox.

Things I need to figure out still, are:

  1. document.caretRangeFromPoint has no equivalent function in Firefox but is being used in Chrome to get the mouseover text. It may be the case that the old way of doing it is still compatible and I will have to port it back in.
  2. Possibility of local SQLite database access? The old addon used Components to get read/write access to a SQLite dictionary packaged with the addon, but I’m not sure it’s possible with WebExtensions.
  3. Login/Session management? I’m pretty sure it’s possible to get user/password and login to HiNative with XHR and proper addon permissions.

WebExtensions and Perakun

09 May 2017

I’ve had this idea of hacking additional functionality onto Perakun for a while now, to make it more useful for taking snippets while reading Japanese. Now that I’m actually diving into it, I’ve learned that there’s a lot of doom and gloom in the Firefox extensions community surrounding the new WebExtensions addon system.

Apparently, the old addon system is going to be phased out by Firefox 57 (slated for November 2017) in favor of WebExtensions. Cited reasons for the move include a need to optimize performance by moving away from the old system (i.e. Gecko) and cross-browser compatibility – WebExtensions addons ostensibly plug into Chrome or any compliant browser with minimal effort. Seeing as how Chrome has 60% market share as of this post, it would be nice to write an extension that can reach that many more users. But developers protest, saying that WebExtensions is still a couple of years out from being even feature-equivalent with what the current addon system offers. The future of some big-name addons beyond the end of this year is unclear. Personally, I am worried about what will happen to my Tree Style Tabs and TiddlyFox support.

But technology trends aside, I really want to take this chance to try my hand at modifying a WebExtensions addon. Chinese Perakun is available in Chrome, but conspicuously, the Japanese version is missing, so I have a straightforward goal:

  1. Clone the Chinese Perakun WebExtensions project and make it fully cross-platform. My test will be that it installs on Chrome and Firefox with no modification.
  2. Create a fork that supports Japanese.
  3. Once the Japanese addon is ready, extend it to be able to post to a Rails app.

I’ve already made headway with my first goal by setting up the cperakun repository. I’ll post a writeup when I’m able to get it up and running in Firefox.

First Android App

23 May 2015 / Filed under Writeup

I’ve finally gotten around to scratching this itch I’ve had and writing this Android app to lookup how to type Chinese characters in Cangjie.

Cangjie is a graphological input method, meaning you type out the character the way it looks rather than how it sounds, like with some Chinese input methods like Pinyin or with Japanese. Knowing Cangjie is handy for studying written language where you don’t know how a character is pronounced. You just decompose the character according to some rules and type it in. The problem is there are a lot of exceptions and tricky cases where you might get stuck. Well, a lot of times, characters reuse components, and you might know how a similar character is pronounced and so you can input it with another method, but at the same time you want to remind yourself of how to key it in with Cangjie. This is the problem I wanted to solve.

Here’s an example character I can easily key in in Japanese. I want to remind myself how to type it with Cangjie…


You take the code and you key it in with the Cangjie keyboard. Practice makes perfect.

I thought this was a simple enough app to get into Android with, so I started off with the Android starter tutorials. The first tutorial gave me pretty much everything I needed as far as UI – then I added a SearchView to the action bar.

The hard part was setting up the database. I decided to piggyback off masaruyokoi’s ChineseCharConverter project. He used Ruby to build a tokyocabinet database to hash Chinese characters to their pronunciations and their Cangjie codes. tokyocabinet is nice and simple database written in C that also has Java bindings, so my plan was to build it for Android and ship the library and the database file with my Android application and access it from there.

That did not really go according to plan.

I really had a tough time figuring out how to properly cross-compile the library. First of all, I didn’t get the Android NDK toolset right at the start but rather downloaded a generic gcc package for aarch64 (this is appropriate for the arm64-v8a architecture of the Samsung S6 that I’m working with). But after I realized that I needed it and got it, I just focused on manually building the library rather than bother with an NDK setup. I just focused on my phone, but NDK contains all the toolchains to support all the Android architectures. I will want to go back and set up the C sources with a proper NDK build as part of the Android project itself. It looks like white-gecko did this conversion a while ago, so I’ll have to refer to his setup.

There was a dependency on bzip2, and I had to build the library itself along with the Java bindings. bzip2 was a simple Makefile project, while tokyocabinet used autoconf. It was quite tricky to configure and I had to tweak the resulting Makefiles. For instance, I had to make sure the soname did not have the version number at the end, otherwise the dynamic libraries can’t resolve and I get a java.lang.UnsatisfiedLinkError from the application. Here are assorted links that helped me figure out how to get it built:

So I was able to get the prebuilt library and database shipped on the phone, but I was still having trouble because I couldn’t open the database. It took me a while trying different methods and locations of storing assets/resources, all the while wondering if I had a permission issue. Android takes care to abstract you from dealing with Files directly, but tokyocabinet actually wants you to give it a File to load its database from. In the end, after asserting I had permission to write, and I’d written a test database on the phone and sent it back to my desktop, I finally realized that the database itself was binary-incompatible across machines. I had to export what I needed from the database in a portable format and build the database up on the phone.

So I had a fun time with Ruby figuring out how to extract JSON from the HDB. This is when I remembered I had OBS set up and started recording a screencast. You can watch me export and import JSON and setup the database query in the Youtube video. Check the description for some timestamps at events leading up to the screenshots you see in this writeup up above. There’s no commentary in this video, so feel free to mute the music if it’s not your thing. I’ve always thought watching people code is cool, and I want to do this kind of thing more, possibly with commentary or just idle chatter with the mic on.

Anyways, I think I had a pretty fun coding session. Android Studio feels quite nice to work with, and I feel like I have gotten a good warmup with Android. As far as this app goes, I want to add a few more things before I put it up on Github, like search history.

Decide what to do, if its doable, do it, and repeat

10 May 2015 / Filed under Productivity

Recently, I’ve picked up my first smartphone and tweaked my work habits. I am now using a pomodoro timer to get myself to do things. The idea is to set a timer for 25 minutes, work on a task for this amount of time, and then break for 5 minutes. I find that there’s a lot of inertia when I sit down in front of the computer but not quite know what I should commit to working on. I feel like 25 minutes is a good unit of time where you can reasonably get something done, but not too much of a commitment that you can just re-evaluate at the end of that period of time and change your course.

I think the break serves a few purposes other than giving you the opportunity to take an actual break from work. It forces you to put a period at the end of the sentence you are writing, so to speak. If you have a point that you still feel is important to make, you can fast-forward the timer to start the next 25-minute interval so you can keep on working. This was what I ended up doing Tuesday morning before my lab presentation. I wanted to throw in a few introductory slides to illustrate some concepts and spent some pomodori to fix an example. I think after seeing the pomodori go by and my example still not finished, I knew that I had to make a decision to wrap it up and move on. It’s nice that there’s also a long 15 minute break is also available after a four-pomodoro run – this is probably the actual time you take a break, because 5 minutes really isn’t enough time to comfortably walk away and have a chat or anything.

In combination with the pomodoro timer, I use the bullet journal to log my progress. Though it’s an analog system, I find it useful to do it electronically, and I use tiddlywiki with a few useful plugins to do the job. You can check it out from my toolbox repository on github here.

← Older

Newer →

Alex Tsui

Alex Tsui

Computer science grad student at UC Davis, writes C++, enjoys learning languages, and occasionally video games.