Albara's blog

This blog for random codes that I use for solving minor problems or do some small tasks, most likely with no categories! I think it's easy for me to come here when I need to reuse something instead of rewriting it over and over again!



Express.js: Kill Process Running on Port 3000

kill -9 $(lsof -t -i :3000)

Clear the terminal history Raspberry Pi

history -c && history -w

Change Raspberry Pi password without having a password

Temporarily adding init=/bin/sh to the end of /boot/cmdline.txt will force the Rpi to boot to a root level console mode where the password for user pi can be changed. 
console=serial0,115200 console=tty1 root=PARTUUID=d6b6a7aa-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-console init=/bin/sh
Reboot then use the following commands 
mount -o remount, rw /
passwd pi 
sync
exec /sbin/init
Ref

NMAP (Network Mapper) on Raspberry Pi

Using a CentOS: 
sudo yum install -y nmap
Using a Debian (Ubuntu/Raspbian): 
sudo apt-get install -y nmap
Using the MAC Address To Identify a Pi 
sudo nmap -sP 192.168.0.0/24

The default SSH user and password on Raspberry Pi

Using shell script, I copy the following file /var/www/html/index.php to all the folders in the current directory: 
username: pi
password: raspberry
To start SSH service 
sudo service ssh start
To check the status of SSH service 
sudo service ssh status

Copying file to multiple folders

Using shell script, I copy the following file /var/www/html/index.php to all the folders in the current directory: 
for dir in *; do [ -d "$dir" ] && cp -f /var/www/html/index.php "$dir" ; done

Zipping folder in Linux

har is the target folder, har.zip is the zipped file: 
zip -r har.zip har

Copying file to multiple folders

Using shell script, I copy the following file /var/www/html/index.php to all the folders in the current directory: 
for dir in *; do [ -d "$dir" ] && cp -f /var/www/html/index.php "$dir" ; done

Changing the EC2 size

follow the direction here: url: ref (click here)
then use SSH to check the size 
df -h
lsblk

Changing the EC2 type

AWS changing the type :ref (click here)
AWS pricing :ref (click here)
AWS types (CPU and RAM) : ref (click here)
then use SSH to check the size

SSH to AWS EC2

ssh -i aaa.pem ec2-user@1.2.3.4

Changing the owner of the file to apache on EC2 AWS

With this command, Apache server (PHP files) will be able to do changes in the files/directories in the server 
sudo chown -R apache:apache ./file.json
With this command, FileZilla (FTP app) will be able to do changes in the files/directories in the server
sudo chown -R ec2-user:apache ./file.json

Set Different Volume Levels For Each App on Mac pro

Here is an app url: click here

Download and run Tensorflow on EC2 AWS

With these commands, you'll be able to install tensorflow in a virtual environment and run it: 
#---------------------------------------------
#Create a virtual environment:
#---------------------------------------------
python3 -m venv --system-site-packages ./venv
source ./venv/bin/activate  # sh, bash, or zsh
pip install --upgrade pip
#---------------------------------------------
#Install TensorFlow
#---------------------------------------------
pip install --upgrade tensorflow
deactivate
#---------------------------------------------
#run python file out of the venv
#---------------------------------------------
./venv/bin/python file.py

Get MAC address of rasperrypi for both eth and wlan

With these commands, you'll be able to get the MAC address of eth0 and wlan0 and print them in whatever text format you wish: 
IFACE_E=eth0
read MAC_E </sys/class/net/$IFACE_E/address
IFACE_W=wlan0
read MAC_W </sys/class/net/$IFACE_W/address
MACX=${IFACE_E}${MAC_E}${IFACE_W}${MAC_W}
MACX2=$(echo $MACX | sed 's/[:"]//g')
echo $MACX2

Camera does not work on macbook-Pro

Run the following command 
sudo killall VDCAssistant

Raspberry Pi: to find the version e.g, 3 or 4

Run the following command 
cat /proc/device-tree/model

Raspberry Pi: SD Card to img on macbook

Find the disk number 
diskutil list
run the command with the disk number (for example here is the disk2): 
sudo dd if=/dev/rdisk2 of=aaa.img bs=1m

Raspberry Pi: img to SD Card on macbook

Find the disk number 
diskutil list
Unmount the disk: 
diskutil unmountDisk /dev/disk2
run the command with the disk number (for example here is the disk2): 
sudo dd bs=1m if=aaa.img of=/dev/rdisk2
Eject the disk. To safely remove the SD card: 
diskutil eject /dev/rdisk2

PHP: To read the csv file without having to deal with new line issue

Here is the code, the csv file will be save in array: $csv 
$filename="aaa.csv";
$file = fopen($filename,"r");
while(! feof($file))
  {
  $csv[]=fgetcsv($file);
  }
fclose($file);

Apahce: Increase the HTTP request size (GET).

First, open the file: 
sudo vim /etc/httpd/conf/httpd.conf
Add the limitationm line (by bytes) 
LimitRequestBody 10000000000
LimitRequestLine 10000000000
LimitRequestFieldSize 10000000000
Then restart the server 
sudo service httpd restart

PHP: Increase the HTTP request size (POST).

First, open the file: 
sudo vim /etc/php.ini
Add the limitation size, or put 0 if you dont want to have any limitation. 
post_max_size = 0

PHP: file_get_contents doesn't work anymore on AWS EC2.

You've to add ssl verify array: 
$arrContextOptions=array("ssl"=>array("verify_peer"=>false,"verify_peer_name"=>false,),);
Then you could call the file_get_contents 
$output = file_get_contents('http://website.com', false, stream_context_create($arrContextOptions));

PHP: file_get_contents doesn't work anymore on AWS EC2.

You've to add ssl verify array: 
$arrContextOptions=array("ssl"=>array("verify_peer"=>false,"verify_peer_name"=>false,),);
Then you could call the file_get_contents 
$output = file_get_contents('http://website.com', false, stream_context_create($arrContextOptions));

Shell: run in the background without saving the log file: nohup.out:

Note first add the follwing line in the sh file: 
#!/bin/bash
Then give the following permsion te the sh file: 
sudo chmod +x aaa.sh
ex. if aaa.sh was your sh file you want to run then the format will be: 
sudo nohup sh aaa.sh >/dev/null 2>&1 &

Shell: Get the size of folder in a path:

ex. if you want to see the size of folders in the home directory: 
sudo du -h --max-depth=1 /home/

Shell: Infinite loop to kill a process by name:

ex. tee is a keyword for a command (replace it with any other word) 
while true ; do sudo pkill -f tee ; done

Pyhton: Create empty array

Here is the source code: 
import numpy as np
np.array([])

EC2: Create swap area

Ref: click here If swapfile already exist 
sudo swapoff -a
The swap file is 4 GB (128 MB x 32): 
sudo dd if=/dev/zero of=/swapfile bs=128M count=32
Update the read and write permissions 
sudo chmod 600 /swapfile
Set up the swap area: 
sudo mkswap /swapfile
Make the swap file available 
sudo swapon /swapfile
Verify 
sudo swapon -s
Start the swap file at boot 
sudo vi /etc/fstab
Add the following line at the end 
/swapfile swap swap defaults 0 0

Linux: Run script in the background (even after exiting the terminal)

Using nohup command you can run a shell script file as follows: 
nohup sh ./cut-all.sh &

Linux: print a size of directory

If the target directory is folder, here is the command: 
sudo du -sh ./folderA

Python3: Virtual environment

Create: 
python3 -m venv ./venv
Activate: 
source ./venv/bin/activate

Python: check if a URL (file, image,page, etc. exist or not without loading the whole file)

import requests

def is_there(path):
    r = requests.head(path)
    return r.status_code == requests.codes.ok

Python/pandas: Mode with a "group by" in pandas

Let's assume you have a pandas data frame df and you want to group by column A and calculate the mode of column B then calculate the mean of B afterward. 
import numpy as np
def mode(a):
    u, c = np.unique(a, return_counts=True)
    return u[c.argmax()]
#####
g=pd.DataFrame(df[['A','B']].groupby(['A']).apply(mode) )
g.columns=["B"]
XMODE=g['B'].mean()
Ref: click here

Python/pandas/matplotlib: Fill the area between 2 pandas curves

Let's assume you have a pandas data frame df and you want to fill the area between columns A and B. With a grey color and transparency of 0.5. 
plt.fill_between(df.index, df['A'],df['B'], interpolate=True, color='grey', alpha=0.5)
Ref: click here

Linux: copy directory dir1 to another one dir2 without losing the contents in the target directory

rsync -r dir1/ dir2

Python/Pandas: filter a columns of data-frame based on contains a partial of string

Let's assume we have a data-frame A and a column called 'user'. Let's say we want to filter the users that contain 1234 
B=A[A['user'].str.contains("1234")]
If we want the once that doesn't contain 1234 
B=A[~A['user'].str.contains("1234")]

Linux: go to the end of the command in htop

To go to the beginning 
 Ctrl+A
To go the end
 Ctrl+E

Linux: protect folder by cookies

Edit the following file: 
sudo vim /etc/httpd/conf/httpd.conf
Add the following commandL 

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_COOKIE} !user=admin [NC]
RewriteRule ^ https://example.net/error.php [NC,L]
Then restart the apache server 
sudo /etc/init.d/httpd restart

Python/panda: read csv using read_csv via https url

Use the follwoing commandas: 
import io
import requests
path="https://example.com/a.csv"
s=requests.get(path).content
cont=io.StringIO(s.decode('utf-8'))
dfx=pd.read_csv(cont,header=None)

AWS/EC2: crontab

To edit the root crontab 
sudo vim /etc/crontab

AWS/EC2: certbot & Let's Encrypt certificates

Follow the instruction in the following link: click here 
sudo yum update
sudo yum install python3 python3-venv libaugeas0
sudo python3 -m venv /opt/certbot/
sudo /opt/certbot/bin/pip install --upgrade pip
sudo /opt/certbot/bin/pip install certbot certbot-apache
sudo ln -s /opt/certbot/bin/certbot /usr/bin/certbot
Certbot edit the apache configuration automatically: 
sudo certbot --apache
Or, just get a certificate: 
sudo certbot certonly --apache
Auto renew: 
echo "0 0,12 * * * root /opt/certbot/bin/python -c 'import random; import time; time.sleep(random.random() * 3600)' && sudo certbot renew -q" | sudo tee -a /etc/crontab > /dev/null

AWS/EC2: set up an EC2/Apache/PHP/MySQL

Step 1 (set up an EC2 instance)

Go to: https://aws.amazon.com/
Create an account (you could use your Amazon account)
Create an EC2 instance.

Step 2 (access the EC2 instance via SSH)

(1) From the Command line we have to change the permission of the Key to make it private: 
chmod 400 ttest.pem
(2) Then we will need to use the following info:
Username: ec2-user

Public DNS: *.*.amazonaws.com        [or you could use the IP instead]

Key: test.pem
(3) From the command line make the SSH connection 
ssh -i ttest.pem ec2-user@*.*.amazonaws.com

Step 3 (set up Apache/PHP/MySql)

(0) you may update the packages 
sudo yum update –y
#sudo yum update
(1) download the PHP package 
#sudo yum install -y httpd24 php56 mysql55-server php56-mysqlnd
sudo yum install -y httpd24 php70 mysql56-server php70-mysqlnd
(2) Start the HTTP service 
sudo service httpd start
#sudo chkconfig httpd on
#chkconfig --list httpd
(3) Go the folder of www (online) 
cd /var/www/
(4) give permission to add/delete and edit for the ec2-user 
sudo chown -R ec2-user:apache /var/www

AWS/EC2: Step 4 (setup mysql_secure_installation)

(1) Start MySQL server. 
sudo service mysqld start
(2) Run mysql_secure_installation. 
sudo mysql_secure_installation
(3) Setup password for example 
username: root

Password: 123456
(4) In case you want to stop MySql 
sudo service mysqld stop
(5) Auto run when setup server 
sudo chkconfig mysqld on

AWS/EC2: Step 5 (Install phpMyAdmin) 

sudo yum install php70-mbstring.x86_64 php70-zip.x86_64 -y
sudo service httpd restart
cd /var/www/html
wget https://www.phpmyadmin.net/downloads/phpMyAdmin-latest-all-languages.tar.gz
mkdir phpMyAdmin && tar -xvzf phpMyAdmin-latest-all-languages.tar.gz -C phpMyAdmin --strip-components 1
rm phpMyAdmin-latest-all-languages.tar.gz
sudo service mysqld start
To access mysql: 
mysql -u root -p

AWS/EC2: cinstall ffmpeg

Follow the instruction: 
sudo cd /usr/local/bin
sudo mkdir ffmpeg2
sudo cd ffmpeg2
sudo wget https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz # https://johnvansickle.com/ffmpeg/
sudo tar xvf ffmpeg-git-amd64-static.tar.xz
sudo mv ffmpeg-git-20220722-amd64-static/ ./ffmpeg
sudo cd ..
sudo rm -r ffmpeg2
sudo cd  ffmpeg
./ffmpeg -version
sudo ln -s /usr/local/bin/ffmpeg/ffmpeg /usr/bin/ffmpeg
Then you could just use the following command from anywhere: Auto renew: 
sudo ffmpeg -i vvv.mov vvv.mp4
Ref: 1 and 2

AWS/EC2: remove http logs

sudo rm -r /var/log/httpd/

RPI: change resultion

Here is a configuratio as an example 
xrandr --output HDMI-1 --mode 1920x1080

RPI: change orientaion and resultion

Here is a configuratio as an example 
nano ~/.profile
# to find HDMI-1 or other run: xrandr
# then run the command with the resution you want: cvt 1920 1080 60
sudo xrandr --newmode "1920x1080_60.00"  173.00  1920 2048 2248 2576  1080 1083 1088 1120 -hsync +vsync
sudo xrandr --addmode HDMI-1 "1920x1080_60.00"
sudo xrandr --output HDMI-1 --rotate right
Ref1 Ref2 Ref3

RPI: trun the screen on or off

On (and rotate it right) 
xrandr --output HDMI-1 --auto --rotate right
Off 
xrandr --output HDMI-1 --off
Ref

Rasberry PI/: Rotate LCD touch screen 7in 90degree to the right

Rotate display 
sudo nano /boot/config.txt
display_lcd_rotate=2
Rotate touch 
# create sh file (e.g screen.sh) and put the command in it
xinput set-prop 'raspberrypi-ts' 'Coordinate Transformation Matrix' 0 1 0 -1 0 1 0 0 1
Open autostart and run the file from it 
sudo nano /etc/xdg/lxsession/LXDE-pi/autostart
sudo screen.sh
Here is the rest angels: 
Rotate left 90 = xinput set-prop 'raspberrypi-ts' 'Coordinate Transformation Matrix' 0 -1 1 1 0 0 0 0 1
Rotate 180 = xinput set-prop 'raspberrypi-ts' 'Coordinate Transformation Matrix' -1 0 1 0 -1 1 0 0 1
Rotate right 90 = xinput set-prop 'raspberrypi-ts' 'Coordinate Transformation Matrix' 0 1 0 -1 0 1 0 0 1
Return to upright = xinput set-prop 'raspberrypi-ts' 'Coordinate Transformation Matrix' 1 0 0 0 1 0 0 0 1
Ref1 Ref2

AWS/Apache: check the url requests

Open config file 
sudo vim /etc/httpd/conf/httpd.conf
Put the following, and change example.com to your domain 

    SetHandler server-status
    Require host example.net
    AllowOverride None
    # Allow open access:
    Require all granted
Then restart apache 
sudo service httpd restart

Rpi4: black monitor issue

When the moniotr HDMI does not display anything except black eventhoug the boot works fine.
Open config file 
sudo nano /boot/conifg.txt
Uncomment the followig line 
hdmi_safe=1
Then restart the device 
reboot

Linx: find file by type in subfolders

When the moniotr HDMI does not display anything except black eventhoug the boot works fine.
Open config file 
find . -name "*.csv"
Ref: click here

Linx: delete file by type in subfolders

When the moniotr HDMI does not display anything except black eventhoug the boot works fine.
Open config file 
find . -name "*.csv" -type f -delete
Ref: click here

AWS EC2: Increase execution time for 1h

First update the php.ini file 
sudo nano /etc/php.ini
update the max_input_time: 
max_input_time = 3600
Second update the httpd 
sudo nano /etc/httpd/conf/httpd.conf
Add this the following line right after the "Listen 80" : 
Timeout 3600

sudo systemctl restart httpd

AWS EC2: Find running python files

ps aux | grep ".py"

AWS EC2: Stop running path

sudo pkill -9 -f "/example/venv/bin/python3"