VirtualBox, Ubuntu Guest, Shared folders

Assuming your Ubuntu guest username is USERNAME:

  1. Make sure you have already installed the VirtualBox Guest Additions in the Ubuntu guest machine

  2. From the Ubuntu guest: sudo adduser USERNAME vboxsf

  3. Using the VirtualBox UI (Devices->Shared Folder Settings…) add the folder in the hosting machine that you wish to share.

  4. Restart the Ubuntu guest! (wasted too much time on that…)

  5. ls /media/ – you’ll find your shared folder there. You might wish to mount it to a different directory on your Ubuntu guest

Sniffing HTTP and HTTPS traffic to understand app protocols

I needed to understand the protocol between an iPhone application and the server. There was no written documentation and the company that owned the application wanted me to build something based on the protocol.

The first thing I tried was using squid as a proxy to intercept the protocol between the iPhone application and the server (click the info icon on the WiFi connection and set the HTTP proxy to the IP and port where the proxy runs on the machine that does the sniffing).

This almost worked. Almost, because squid was not able to proxy the HTTPS traffic to Facebook, which was necessary before connecting to the application server.

So, at this point I installed mitmproxy, a man-in-the-middle proxy that can sniff both HTTP and HTTPS traffic.
To sniff HTTPS traffic, mitmproxy on one hand presents to the client (the iPhone application in this case) a fake Facebook certificate, and on the other hand, presents itself to Facebook as if it were the original client.

For the client to accept the fake Facebook certificate, it has to be both known to the mitmproxy and also installed as a trusted certificate on the device.

For the certificate to be known to mitmproxy it should be created and passed as a parameter. This is achieved as follows (thanks to Nathan):

> openssl genrsa -out fake_facebook_ca.key 2048
> openssl req -new -x509 -key fake_facebook_ca.key -out fake_facebook_ca.crt

   ... No need to fill out optional fields ...
   Common Name (eg, YOUR name) []:*.facebook.com  (domain you want to MITM here)
   ...

> cat fake_facebook_ca.key fake_facebook_ca.crt > fake_facebook_ca.pem 
> mitmproxy --cert=fake_facebook_ca.pem

However, the above is not enough since the client has to trust this fake certificate. To do that, send the fake_facebook_ca.crt file as an attachment to an email and open this attachment on the iPhone on which the application that you want to sniff is located. You’ll be prompted whether you authorise installing and trusting this certificate. After confirming, open the application and you’ll be able too sniff the entire HTTP/HTTPS session on the mitmproxy console.

Speeding up MySQL imports from mysqldmp

If you need to import a very large SQL dump file to MySQL, the following should speed things up considerably:

At the begining of the SQL dump file, add:

SET autocommit=0;
SET unique_checks=0;
SET foreign_key_checks=0;

and at the end:

COMMIT;

Then proceed as usual with the import, i.e

mysql> source myfile.sql

Errors installing Ubuntu guest additions on VirtualBox

If you mount the guest additions and see a warning about missing kernel headers, and the additions don’t seem to work, and then after trying to launch the guest additions CD again you get VERR_PDM_MEDIA_LOCKED error and everything starts going downhill from there – this is the solution that worked for me:

1. Reinstall Ubuntu

2. Before installing the guest additions:

sudo apt-get update
sudo apt-get install build-essential linux-headers-`uname -r` dkms

3. Now launch the guest additions CD

Simple CSV read and write using Lua

Update Dec 5th 2014: Fixed and updated the code.
Update Mar 9th 2017: Updated the code to handle null values in CSV.

This is a simplified Lua script to read and write CSV files. It assumes the separator character does not exist in one of the values.

I don’t know why I couldn’t find a simple code snippet to do this on the net, but sometimes it’s just faster to write it than to search for it. Hopefully this will save you that time too:

--[[
-------------------------------
Save this file as simplecsv.lua
-------------------------------

Example use: Suppose file csv1.txt is:

1.23,70,hello
there,9.81,102
x,y,,z
8,1.243,test

Then the following
-------------------------------------
local csvfile = require "simplecsv"
local m = csvfile.read('./csv1.txt') -- read file csv1.txt to matrix m
print(m[2][3])                       -- display element in row 2 column 3 (102)
m[1][3] = 'changed'                  -- change element in row 1 column 3
m[2][3] = 123.45                     -- change element in row 2 column 3
csvfile.write('./csv2.txt', m)       -- write matrix to file csv2.txt
-------------------------------------

will produce file csv2.txt with the contents:

1.23,70,changed
there,9.81,123.45
x,y,,z
8,1.243,test

the read method takes 4 parameters:
path: the path of the CSV file to read - mandatory
sep: the separator character of the fields. Optionsl, defaults to ','
tonum: whether to convert fields to numbers if possible. Optional. Defaults to true
null: what value should null fields get. Optional. defaults to ''
]]

module(..., package.seeall)

---------------------------------------------------------------------
function string:split(sSeparator, nMax, bRegexp)
    if sSeparator == '' then
        sSeparator = ','
    end

    if nMax and nMax < 1 then
        nMax = nil
    end

    local aRecord = {}

    if self:len() > 0 then
        local bPlain = not bRegexp
        nMax = nMax or -1

        local nField, nStart = 1, 1
        local nFirst,nLast = self:find(sSeparator, nStart, bPlain)
        while nFirst and nMax ~= 0 do
            aRecord[nField] = self:sub(nStart, nFirst-1)
            nField = nField+1
            nStart = nLast+1
            nFirst,nLast = self:find(sSeparator, nStart, bPlain)
            nMax = nMax-1
        end
        aRecord[nField] = self:sub(nStart)
    end

    return aRecord
end

---------------------------------------------------------------------
function read(path, sep, tonum, null)
    tonum = tonum or true
    sep = sep or ','
    null = null or ''
    local csvFile = {}
    local file = assert(io.open(path, "r"))
    for line in file:lines() do
        fields = line:split(sep)
        if tonum then -- convert numeric fields to numbers
            for i=1,#fields do
                local field = fields[i]
                if field == '' then
                    field = null
                end
                fields[i] = tonumber(field) or field
            end
        end
        table.insert(csvFile, fields)
    end
    file:close()
    return csvFile
end

---------------------------------------------------------------------
function write(path, data, sep)
    sep = sep or ','
    local file = assert(io.open(path, "w"))
    for i=1,#data do
        for j=1,#data[i] do
            if j>1 then file:write(sep) end
            file:write(data[i][j])
        end
        file:write('\n')
    end
    file:close()
end

---------------------------------------------------------------------

How I wasted an entire day on wrong MIME type

Problem description:

  • External CSS file required by HTML was being downloaded from the server but not being applied to the Web page
  • To add to the frustration, if the Web page (which, as mentioned, was rendered without the CSS) was then saved locally from the browser and then that file opened in the browser, it would render just fine
  • Removing the <!doctype html> from the first line of the HTML file seemed to solve the problem and cause the CSS to apply to the HTML, but I wasn’t prepared to do that since it seemed that this is a symptom of a problem rather than a solution.

After many searches and trials, I looked at the Network tab in Chrome browser and although all the CSS files were being downloaded, their MIME type was text/plain. The reason is that the Web server (in my case nginx) was sending out the wrong MIME type with the css files, and <!doctype html> apparently does not like that which causes the files not to be parsed or applied.

The solution in my case was to add

location ~ .css {
    add_header  Content-Type    text/css;
}
location ~ .js {
    add_header  Content-Type    application/x-javascript;
}

under the server { section, since for some reason

include  /usr/local/openresty/nginx/conf/mime.types;

wasn’t doing it’s job.

Taking ScreenShots of your Mobile App

Android

On Android, you first need to define an emulator if you don’t have one. You do this by launching Eclipse ADT > Window > Android Virtual Device Manager.

One you have defined your emulator, you can check your Cordova app on it by:

cordova build android

followed by

cordova emulate android

It takes ages for the emulator to launch with your app so be patient…
After your app loads and you want to take a screen shot from the emulator, go back to the eclipse window and from the menu:

Window > Show View > Other. In the newly opened dialog box, under Android category, select Devices.

Now on the Devices panel, Click the camera button – you’ll figure it out from there.

iOS

On iOS things are simpler – run the app in the emulator by opening the project in XCode, selecting the type of device emulator and running your app. Command-S will save a screen shot on your Mac desktop.

Some Cordova 3.3 gotchas (there are more)

Using the inappbrowser plugin:

what should be in your config.xml is:

    <feature name="InAppBrowser"> 
       <param name="android-package" value="org.apache.cordova.inappbrowser.InAppBrowser"/>
    </feature>

and not

    <feature name="InAppBrowser"> 
       <param name="android-package" value="org.apache.cordova.InAppBrowser"/>
    </feature>

Also, if you want to use the back button handler on Android, make sure you have the following in your config.xml

    <feature name="App">
        <param name="android-package" value="org.apache.cordova.App" />
    </feature>

Another issue (not related to Cordova gotchas) is adb devices returning a status of offline for recognised Android devices connected via USB. There can be multiple reasons that cause this, but what worked in my case was:

  1. kill the adb service running on your PC
  2. Disable USB debugging on your Android device and remove the USB from your PC
  3. Restart your Android device
  4. Enable USB debugging on your Android device
  5. Connect the USB cable from your Android device to your PC
  6. run ‘adb devices’ from the terminal

Playing sound with Phonegap/Cordova

Let me save you a lot of time: The Cordova Media plugin (org.apache.cordova.media) does not work well on Android and Windows Phone 8, at least in the version released with Cordova 3.1 (and I don’t see any indication of a change in the latest release either).

On Android I assume media resources are not released by the plugin since after a few plays, the audio does not play anymore unless the application is restarted. On Windows Phone 8, the audio sort of plays, but causes the application to behave very strangely (unrelated Javascript callbacks are not invoked, probably due to threading issues and extremely sluggish behaviour in general)

What I did to play audio in a Cordova application was the following:

For Android, iPhone – use the following plugin:
cordova plugin add https://github.com/triceam/LowLatencyAudio

For Windows Phone 8, create a static HTML5 audio tag, get a pointer to it from the DOM (e.g, by getElementById) and just use the ‘play’ method on the returned object whenever you want to play the audio.