updating all WordPress plugins from the command line ∞
wp plugin update --all
Use specific aws credentials profile with python/boto3 ∞
from here
boto3.setup_default_session(profile_name='myprofile')
Copy file from s3 to local filesystem with python/boto3 ∞
from here
$result = $s3->getObject(array( 'Bucket' => $bucket, 'Key' => $key, 'SaveAs' => $path . $model->file_name, ));
updating pip/pip3 ∞
Sometimes, when you run into instances where the current pip is having problems installing a package and the usual method of upgrading pip doesn’t work (i.e pip install --upgrade pip ) then try the following:
curl https://bootstrap.pypa.io/get-pip.py | python
from here
Downloading AWS lambda zip package ∞
The following is an example of how to download an aws lambda zip package that was previously uploaded as a lambda function (assuming ~/.aws/credentials contains a profile named lambda_user)
aws lambda get-function --region us-east-1 --function-name lambda_pdf_func --profile lambda_user
The reply (if no errors) will be a JSON string. In it, look for the key “Code”, and within “Code”, look for “Location”. The value of “Location” is a link from which you can download the current zip package that corresponds to the function (in this case, lambda_pdf_func)
Open MySQL running on EC2 to remote connections ∞
First, let it be on the record that I don’t think this is a good idea security-wise, but for those that want/have to ignore this advice, here are the steps:
- Open TCP port 3306 using your AWS EC2 console
- edit either
/etc/mysql/my.cnfor/etc/mysql/mysql.conf.d/mysqld.cnfand search for the stringbind-address - if
bind-addressis 127.0.0.1 change it to 0.0.0.0 (this will enable the mysql process to accept remote connections) - restart the mysql service (via
sudo /etc/init.d/mysql restartor whatever method you prefer) -
login to mysql with administrator privileges (e.g as root) and create a new mysql user that can access your database remotely – the following example assumes the username is
remoteuserand the database you want to user to have access to ismydatabase:
mysql> CREATE USER 'remoteuser'@'%' IDENTIFIED BY 'your_password'; mysql> GRANT ALL PRIVILEGES ON mydatabase.* TO 'remoteuser'@'%' WITH GRANT OPTION;
finally, to check that it works, from your local command line, assuming your database runs at your.dbserver.com:
mysql -u remoteuser -h your.dbserver.com -pyour_password
Some ffmpeg tricks ∞
Convert VOB file to mp4 using ffmpeg:
ffmpeg -i outss.VOB -acodec libmp3lame -ac 2 -ar 22050 video.mp4
___
To extract only the video track to a new VOB file (VOB is the container, the codec is mpeg2):
First, find the channel where the video stream is located using:
ffmpeg -i video1.VOB
The list of existing streams in the file will be returned. Note what follows the ‘Stream #’ in the video stream. If for example the video stream you’re interested in is 0:1 then:
ffmpeg -i video1.VOB -map 0:1 -vcodec copy silent_video.VOB
___
To extract a stream between two points in time:
ffmpeg -i bigfile.VOB -ss 00:28:00 -to 00:35:30 -c copy section.VOB
the above will extract the section starting at minute 28 in the video until minute 35:30 into the video
___
To fix timestamps gone haywire (you can tell by VLC not showing the current playing time or ffplay not being able to skip forward). Taken from here
ffmpeg -err_detect ignore_err -i video.mkv -c copy video_fixed.mkv
___
To change properties, for example increase brightness by a factor of two – first try the values using ffplay, e.g
ffplay -vf lutyuv=y=val*2 myvideo.VOB
Then create a modified video with these parameters:
ffmpeg -i myvideo.VOB -vf lutyuv=y=val*2 brighter.VOB
More examples here
Good cross platform PDF utils ∞
pdf to text, pdf to images, pdf to html, pdf info – great command line utilities and open source:
http://www.xpdfreader.com/download.html
pdf2ppm is also good but hard to find a version that works on Windows
Iterating through maps (tables/hash tables/dictionaries/associative arrays) ∞
If you also find yourself making use of more programming languages and software technologies than you care to count, then you probably also forget from time to time the syntax for doing things. A quick reminder:
Lua
Lua doesn’t distinguish (at least not externally) between dictionaries and arrays
local hash={ 1, 'two', three=3, ['& four']='quatro' } for i=1,4 do print( hash[i] ) end print('n') for k,v in pairs(hash) do print(k,v) end
1 two nil nil 1 1 2 two three 3 & four quatro
Python
hash={ 1:1, 2:'two', 'three':3, '& four':'quatro' } for i in range(0,4): if i in hash: print(hash[i]) else: print("no key") print("") for i in hash: print("%st%s" % (i,hash[i]))
no key 1 two no key 1 1 2 two & four quatro three 3
JavaScript
var hash={ 1:1, 2:'two', 'three':3, '& four':'quatro' } for (var i=0; i<4; i++) if (i in hash) console.log(hash[i]); else console.log("no key"); console.log('') for (i in hash) console.log( i+"t"+hash[i] )
no key 1 two key 1 1 2 two three 3 & four quatro
PHP
<!--?php $hash = array( 1=-->1, 2=>'two', 'three'=>3, 'four'=>'quatro' ); for ($i=0; $i<4; $i++) if (isset($hash[$i])) print("$hash[$i]n"); else print("no keyn"); print("n"); foreach ( $hash as $key => $value ) printf( "$keyt$valuen" ); ?>
no key 1 two no key 1 1 2 two three 3 four quatro
Perl
my %hash = ( 1=>1, 2=>'two', 'three'=>3, 'four key'=>'quatro' ); for ($i=0; $i<4; $i++) { if (exists $hash{$i}) { print("$hash{$i}n"); } else { print("no keyn"); } } print("n"); while ( ($key, $value) = each %hash ) { print("$keyt$hash{$key}n"); }
no key 1 two no key 2 two 1 1 three 3 four quatro
Ubuntu – check which processes are listening on which addresses/ports ∞
sudo netstat -plnt
Free Mac utility for hardware monitoring ∞
istats – CPU temperature, fan speed, battery health and more
Add user to MySQL database ∞
CREATE USER 'newuser'@'localhost' IDENTIFIED BY 'password'; GRANT ALL PRIVILEGES ON * . * TO 'newuser'@'localhost'; FLUSH PRIVILEGES;
change root password of MySQL ∞
mysqladmin -u root -p"<old_pass>" password <new_pass>
import csv to table ∞
mysqlimport -v --local --fields-enclosed-by='"' --fields-escaped-by='\' --fields-terminated-by=',' --lines-terminated-by='\n' -u<username> -p<password> -h localhost <dbname> '<filename_identical_to_tablename>.csv'
Find hard disk space hogs ∞
Need to see what’s using up all that space on your hard drive?
cd / sudo du -hd1
and drill down from there…
Quickly searching through bash history ∞
If you know your way around the bash command line, you know that <Ctrl>r searches the last command in your history that started with the letters that you proceed type afterwards. While that is nice, it’s not always that practical since many times you want to find the command that starts the same way but was previous to the one that <Ctrl>r found.
one way is to combine history with grep, but a better way is to add the following lines to your .bashrc (or .bash_profile on Mac):
if [[ $- =~ i ]]; then # only do this if we are in an interactive shell bind '"e[A":history-search-backward' bind '"e[B":history-search-forward' fi
Next time you open a bash window and want to search for a previous command, simply type the first letters and hit the up arrow. Each additional press of the up arrow will bring a matching command further up the history.
Viewing root directories in Finder window on OS/X ∞
For example, if you want that the /usr directory will be visible:
sudo chflags nohidden / sudo chflags nohidden /usr
If you want to hide the /usr directory again:
sudo chflags hidden /usr
Codesigning gdb on OS/X ∞
Taken from the GNU docs
The Darwin Kernel requires the debugger to have special permissions
before it is allowed to control other processes. These permissions
are granted by codesigning the GDB executable. Without these
permissions, the debugger will report error messages such as:
Starting program: /x/y/foo Unable to find Mach task port for process-id 28885: (os/kern) failure (0x5). (please check gdb is codesigned - see taskgated(8))
Codesigning requires a certificate. The following procedure explains
how to create one:
- Start the Keychain Access application (in
/Applications/Utilities/Keychain Access.app)
- Select the Keychain Access -> Certificate Assistant ->
Create a Certificate… menu
- Then:
- Choose a name for the new certificate (this procedure will use
“gdb-cert” as an example)
- Set “Identity Type” to “Self Signed Root”
- Set “Certificate Type” to “Code Signing”
- Activate the “Let me override defaults” option
- Choose a name for the new certificate (this procedure will use
- Click several times on “Continue” until the “Specify a Location
For The Certificate” screen appears, then set “Keychain” to “System”
- Click on “Continue” until the certificate is created
- Finally, in the view, double-click on the new certificate,
and set “When using this certificate” to “Always Trust”
- Exit the Keychain Access application and restart the computer
(this is unfortunately required)
Once a certificate has been created, the debugger can be codesigned
as follow. In a Terminal, run the following command…
codesign -f -s "gdb-cert" <gnat_install_prefix>/bin/gdb </gnat_install_prefix>
… where “gdb-cert” should be replaced by the actual certificate
name chosen above, and
the location where you installed GNAT.
Quick HTTP Server for testing stuff ∞
python -m SimpleHTTPServer
set default editor for commands like crontab -e ∞
sudo select-editor
Mac – refresh /etc/hosts configuration ∞
OS X Mountain Lion or Lion
Use the following Terminal command to reset the DNS cache:
sudo killall -HUP mDNSResponder
Mac OS X v10.6
Use the following Terminal command to reset the DNS cache:
sudo dscacheutil -flushcache
Also reset browser cache if necessary.
Unix – Recursively search files ∞
beginning with “myfile” from current path
find . -name "myfile*"
vim tricks to remember ∞
When editing CSV files
-
If the columns are nice and straight throughout the file, you can make use of <ctrl>-v which puts you in column visual mode and lets you mark the columns that you wish to copy/paste/delete etc.
-
If the CSV is less well behaved, you can use regex with capture groups to search and replace, for example:
:%s/^.+,.+,.+,(.+),.+/1/gwill remove the first 3 columns and last column from a CSV file
Extracting bz2 files ∞
taken from here:
First, make sure you have bzip2 installed
apt-get install bzip2
and to do the magic:
tar jxvf phantomjs-1.9.2-linux-i686.tar.bz2
Saving video streams to a file ∞
To save video that is streamed to a Web page as a file, you should first locate the uri that the plugin that renders the stream uses to fetch the media. For example, if the web page uses CastUP technology to stream video then you need to look for the point where the plugin requests the mms://<some-identifier>.wmv
To do this, open the a terminal console window and enter:
tcpdump -A 'tcp port 80' | grep mms
Then, open the web browser at the page where the video is rendered. You should be able to see in the terminal console the references to the mms:// url.
Copy the url and feed it either to VLC or mencoder as specified here.
and just in case the link above will be lost:
using mencoder:
mencoder mms://ip-address/foo.wmv -ovc copy -oac copy -o foo.avi mencoder mms://1.2.3.4/bar.wmv -ovc copy -oac copy -o bar.avi
using VLC:
vlc mms://1.2.3.4/foo/bar --sout=file/ps:output.mpg
sed, awk, tr etc ∞
replace whitespaces between words with a newline:
tr ' ' 'n' < infile > outfile
Converting between text file formats ∞
Suppose you have a file encoded in one text format, say UTF16-LE and you wish to convert it to another format, for example UTF-8
iconv is an amazing piece of work that can convert between many different text encoding formats and is also available for all major platforms.
In general using iconv goes like this
iconv -f from_encoding -t to_encoding input_file.txt > converted_file.txt
where from_encoding/to_encoding are the existing encoding of the file and the encoding you want to convert to respectively.
The list of encodings iconv supports can be viewed bu issuing iconv -l
If you don’t know the current encoding of the input_file.txt in the above example, on unix based systems issue a
file -i input_file.txt
This will show the existing encoding of the file you wish to convert so that you can enter the -f parameter for the conversion
In cases where file -i claims that what you have is a regular text file but gives you no additional information as to the file’s encoding, you should use enca , which is a great tool for discovering the existing text encoding of a file. In most cases all you’ll need to do is (assuming infile.txt is the file whose encoding you want to know):
enca infile.txt
Or, if the above doesn’t work, for multibyte encodings try:
enca -L none infile.txt
Check man enca for more info.
Batch Image Processing ∞
I’ve done this before but already forgot about it. You have a directory full of images and you want to process them all in one shot – either resize, convert format, flip, join, crop, blur etc. What you do is use mogrify. It works like this:
- Make sure you have imagemagick installed. If not then
sudo apt-get install imagemagick -
Then just run mogrify (one member of the ImageMagick suite of tools) on the set of images you want to transform. Note – the images will be overwritten with the new files – make sure you have copies of the images if you use mogrify else use the
converttool.
An example on running mogrify to resize all the jpg files in a directory to a width of 200 pixels (preserving the aspect ratio):
mogrify -resize 200 *.jpg
Create Desktop Launcher/Shortcut ∞
This was removed from the UI in Ubuntu 12.04, but you can still do it as follows:
If you don’t have it already installed, then:
sudo apt-get install gnome-panel
Next, simply
gnome-desktop-item-edit ~/Desktop/ --create-new
(thanks)
Burn a directory to multiple CDs/DVDs ∞
its called dirsplit and is used thus:
Suppose you have a directory with gadzillion files (mp3s, images, whatever), and you want to burn them all on CD/DVD media. It will take a few volumes but you don’t want to start dividing the files into folders of CD/DVD size. For the example say these are images that are stored in /home/myplace/CamUploads/
What you do is:
cd /home/myplace/CamUploads/ dirsplit -s 700M /home/myplace/CamUploads/
The above will create of files with extension .list (vol_1.list, vol_2.list, etc. depending on the total size). These files can be processed to create corresponding iso images for burning on the CD/DVD by issuing:
mkisofs -o vol1.iso -D -r --joliet-long -graft-points -path-list vol_1.list mkisofs -o vol2.iso -D -r --joliet-long -graft-points -path-list vol_2.list
Use your favorite CD burner (read: K3b) to burn the iso images to your CD/DVD media.
Batch deletion of files ∞
When you have files that have a common pattern dispersed throughout a complex directory structure and you wish to delete them, you can use the Linux powerful find command. Among its many parameters are the -exec switch which enables executing a command for every file found.
Suppose you want to delete all the files within a directory structure whose root is at /home/me and that end with _dev.db3. To achieve this simply do the following:
cd /home/me
find . -type f -name "*_dev.db3" -exec rm -f "{}" ;
{} represents the files that are found for the exec switch. It is quoted to avoid surprises when running rm on it. The command after the exec has to be terminated by a ; which itself is escaped by a backslash to avoid additional unintentional shell interpretations.
man find for more information