I took a couple of hours to create an interactive explanation for those who want to get an intuitive grip on the confusion matrix – precision, recall, F1, accuracy etc.

Still need to improve and polish but you can already play with it:

http://nocurve.com/content/confmat.html

To use binaries from within AWS lambda functions, you need to have the compiled binaries available within the lambda package used and for that you sometimes need to compile those yourself. To do that, you can either compile on an Amazon Linux docker image (more information on how to do this here and from the aws docs) or compile the binary on an EC2 instance that uses the same environment as the underlying AWS Lambda execution environment. This latter is what’s described here.

You can launch this EC2 by the AMI mentioned here:

Note that the EC2 instance that will be launched is an Amazon Linux EC2 (Not Debian/Ubuntu) which means that to login you’ll need to use

ssh -i my-private-key.pem ec2-user@the.ipa.ddr.ess

The following table summarizes various user names for different EC2 platforms (taken from here):

Of course, don’t forget to chmod 600 your private key or you’ll get a complaint from ssh…

Once logged in to the Amazon Linux EC2, you’ll want to install the relevant development tools to be able to compile whatever it is you’re compiling. Unlike Debian family of Linux distributions (Debian, Ubuntu, Mint etc.) which use apt/apt-get/dpkg to install packages, Amazon Linux uses the yum package manager (RedHat, Fedora etc.), and while on a Debian system you’d use sudo apt-get install build-essential to install your gcc compiler and other necessary development tools, to achieve more or less the same with yum, enter:

sudo yum groupinstall "Development Tools"

Now you’re ready to start compiling. Hopefully, the next post will explain how to package and launch the compiled binaries and their dependencies. I expect a lot of environment/path definitions, dependency tracking, and other gotchas along the way, but one step at a time…

Everything is a trade-off (of course, that’s actually another Maslow-type hammer, but nevertheless a useful one) and many times (for me at least) a lot of time is wasted on agonizing getting a good understanding what the trade-offs are before committing to an implementation. Of course, part of the reason is that I never want to commit to anything, and want to have the maximum flexibility at all times, which can lead to analysis paralysis – so the first tip from this post is “It doesn’t really matter”. The reason it doesn’t really matter boils down to the wise phrase “it’s far more important to get it going than to get it right”. Of course, one could counter with “there’s never time to do it right but always time to do it over”, and the funny thing is that the conflict between these two phrases is yet another trade-off (this is so meta…) but the bottom line is that while it’s always nice if you choose the correct path from the start, you’ll never be able to avoid mistakes indefinitely and getting into “dear in the headlights” mode from fear of making a mistake is a far worse choice than any other option.

Still, the ability to make a good choice from the start is nice, and therefore I wanted to share an article that helps decide between using AWS Lambda or EC2 for a service you wish to implement.

https://www.trek10.com/blog/lambda-cost/

The problem
Python script invoked from PHP via shell_exec and runs fine when PHP invoked from command line but fails when PHP triggered by browser access.

Reason
PHP, when triggered by a browser access is invoked by web server with the user www-data, while from the command line it is run as user ubuntu.

Attempting to see what happens when running PHP from the command line as user www-data would help to understand why running the script fails

One method to run php as www-data from the command line is to enable a shell for www-data user. This is done by modifying /etc/passwd so that user www-data has a shell (change the existing /usr/sbin/nologin or whatever to /bin/bash or something similar) and then sudo su www-data and try to run the python script again (see this reply for details).

Doing the above, quickly showed that the one of the imports in the python script fails when running it under www-data.
Comparing python3 -m site when running under user www-data vs. when running under ubuntu showed there is a difference in the module search paths.

Adding the missing path found for user ubuntu to user www-data via sys.path.insert was not scalable, nor possible (since the ubuntu user path is inaccessible to the www-data user), so the best way was to install the python modules (in my case, imagehash) in a way that will be accessible to the www-data user

The solution, found here illustrated how this is done

sudo mkdir /var/www/.local
sudo mkdir /var/www/.cache
sudo chown www-data.www-data /var/www/.local
sudo chown www-data.www-data /var/www/.cache
sudo -H -u www-data pip install imagehash

Method #2 ∞

Of course, a simpler alternative to this is to run apache as ubuntu, which will make all the above unnecessary assuming the situation/security requirements enable it, in which case you might want to also change the htdocs directory:

sudo vim /etc/apache2/envvars # change APACHE_RUN_USER and APACHE_RUN_GROUP to ubuntu
cd /etc/apache2/sites-available/
sudo cp 000-default.conf 000-ubuntu.conf
sudo vim 000-ubuntu.conf   # change the path for DocumentRoot
sudo a2dissite 000-default.conf
sudo a2ensite 000-ubuntu.conf

Also,
sudo vim /etc/apache2/apache2.conf

and add the following:

<Directory /home/ubuntu/>
        Options Indexes FollowSymLinks
        AllowOverride None
        Require all granted
</Directory>

Don’t forget to

sudo systemctl restart apache2

or

service apache2 reload

Download the VirtualBox image from https://www.virtualbox.org/wiki/Downloads

Download the 16.04 Ubuntu image for VirtualBox from www.osboxes.org (the credential details for logging in are here: https://www.osboxes.org/faqs/)

Open VirtualBox, click the New icon, Select ‘Linux’ for type, ‘Ubuntu 64-bit’ for version (assuming you downloaded the VirtualBox 64bit vdi), click next a couple of times until you reach the ‘Hard Disk’ section and then click the ‘use an existing virtual hard disk file’ radio button
and browse for the vdi file you downloaded.

Log in using the credentials given (osboxes.org as password). After logging in it is recommended to install the guess additions (to be able to have things like shared folders and shared clipboard). Do this by selecting Devices->Insert Guest additions CD image… from the VirtualBox menu. This will add a CD icon to the unity bar on the left, meaning the guest additions CD has been mounted. Open a terminal window and type the following at the command prompt:
sudo /media/osboxes/VBox_GAs_5.2.8/VBoxLinuxAdditions.run
This will install the guest additions on the Ubuntu guest system. Ignore the #modprobe vboxsf failed# error at the end and restart the system via typing sudo restart at the command line
After successfully installing the guest additions you can eject the CD (right-click the CD icon and select eject)

After logging in again, open the terminal window again and do:
sudo apt-get update
sudo apt-get upgrade

The above update and upgrade will take a few minutes so after it is done, it is recommended to export your image should you want to have run the updated VirtualBox image on another machine without going through the entire process above yet again. Do this by closing the window and selecting ‘Power off the machine’. Next, Open the Oracle VirtualBox manager. Select File->Export Appliance from the menu, select the name of machine and click Next a couple of times. The resulting file/files can then be imported to a different VirtualBox app.

The way to select a column in Sublime Text on Windows is by using + and then pressing the up or down arrow keys. You might get a surprise though if you try it in the form of you entire screen display flipping upside down. Some people don’t even know this is possible to do so they get extra surprised.

This is because on certain Intel graphic cards, there are the hotkeys to trigger certain functions of the graphic card, for example being the functions to flip and rotate the screen (this is handy for instance if you want to work with your screen physically rotated to be in portrait rather than landscape mode and you’d obviously want to render the desktop accordingly). The keys to select a text column in Sublime Text happens to be taken by the hotkey for flipping the screen by the graphic card.

To disable this key combination being hijacked by the graphic card, you can either customize the graphic card to use a different key combination (Ctrl+Alt+F12 > Options) or disable the hotkeys for graphic functions altogether (Ctrl+Alt+F12 > Options > Uncheck “Enable Hot Keys”)

Found the above thanks to a comment here (by L_7337)

Someone sent you an invitation to an event as an .ics file, and you wish to add it to your Google calendar

The first part is to import the file. To do this, click the + icon to the right of the “Add a friends calendar” located on the left side of the page. After clicking the icon, from the available options select ‘Import’

Now, the obvious thing to do would be to just to click the button that says “Select file from your computer”, select the .ics file and click import, but if you would do that, chances are that you’ll see the dreaded:

‘Failed to import events: Could not upload your events because you do not have sufficient access on the target calendar..’

The solution for this is to manually edit the .ics file prior to importing it and replace all occurrences of “UID:” with “UID:X” (without the quotes). After doing this and saving the file, proceed with the import and all should be fine.

Learned this from here

Many people are not aware that since the early days of the simple notepad app that comes bundled with Windows, it had the following undocumented feature: If you enter the text .LOG as the first line of the file, then every time you open the file with notepad, it will append the current date and time to the end of the document and scroll there. This is quite handy when you want a file that keeps track of the time when you added new entries.

I wanted to have the same functionality with Google Docs (with the added benefit of not needing to write .LOG at the beginning of the file). The following script (built via multiple shameless plagiarism from various sources) enables that functionality:

function onOpen() {
  var ui = DocumentApp.getUi();
  // Or FormApp or SpreadsheetApp.
  ui.createMenu('Custom Menu')
      .addItem('Insert Date', 'insertDate')
      .addToUi();

  setCursorToEnd()
  insertDate();
  setCursorToEnd()
}

function setCursorToEnd()
{
  var doc = DocumentApp.getActiveDocument();
  var paragraph = doc.getBody().appendParagraph('');
  var position = doc.newPosition(paragraph, 0);
  doc.setCursor(position);
}

function formatDate()
{
  var date = new Date();
  var datestr = date.getFullYear() + '-'+  
    ('0' + (date.getMonth()+1)).slice(-2) + '-' + 
    ('0' + date.getDate()).slice(-2);  
  var hours = date.getHours();
  var minutes = date.getMinutes();
  var ampm = hours >= 12 ? 'pm' : 'am';
  hours = hours % 12;
  hours = hours ? hours : 12; // the hour '0' should be '12'
  minutes = minutes < 10 ? '0'+minutes : minutes;
  var strTime = hours + ':' + minutes + ' ' + ampm;
  return datestr + "  " + strTime;
}

function insertDate()
{
  var activeDoc = DocumentApp.getActiveDocument();
  var cursor = activeDoc.getCursor();
  if (cursor) {
      var date = formatDate();    
      var element = cursor.insertText(date);
  } else {
      DocumentApp.getUi().alert('Cannot find a cursor in the document.');
  }
}

To enable this for a document, In Google Docs : Tools -> Script Editor, enter and save the above script and then accept the permissions.

Switching between various development environments, to save restart time, I wanted to start various services from OS/X only if they were not already running. Brew does not have a method to know whether a particular service is running (AFAIK), so the following example code parses the list of services returned by brew services list and then enables quickly checking the status of a given service and acting accordingly:

#!/bin/bash
brew services list | awk -F " " '{print $1 "=\"" $2 "\""}' > __tmp__.sh
chmod u+x __tmp__.sh
source ./__tmp__.sh # so that variables are maintained in this shell
rm ./__tmp__.sh

if [ "$php72" == "started" ]; then
        echo "PHP service is up"
fi

if [ "$openresty" == "started" ]; then
        echo "Openresty service is up"
fi

https://github.com/wilfredwee/photon-australis