Albara's blog

This blog for random codes that I use for solving minor problems or do some small tasks, most likely with no categories! I think it's easy for me to come here when I need to reuse something instead of rewriting it over and over again!



Copying file to multiple folders

Using shell script, I copy the following file /var/www/html/index.php to all the folders in the current directory:
for dir in *; do [ -d "$dir" ] && cp -f /var/www/html/index.php "$dir" ; done

Zipping folder in Linux

har is the target folder, har.zip is the zipped file:
zip -r har.zip har

Copying file to multiple folders

Using shell script, I copy the following file /var/www/html/index.php to all the folders in the current directory:
for dir in *; do [ -d "$dir" ] && cp -f /var/www/html/index.php "$dir" ; done

Changing the EC2 size

follow the direction here: url: ref (click here)
then use SSH to check the size
df -h
lsblk

Changing the EC2 type

AWS changing the type :ref (click here)
AWS pricing :ref (click here)
AWS types (CPU and RAM) : ref (click here)
then use SSH to check the size

SSH to AWS EC2

ssh -i aaa.pem ec2-user@1.2.3.4

Changing the owner of the file to apache on EC2 AWS

With this command, Apache server (PHP files) will be able to do changes in the files/directories in the server
sudo chown -R apache:apache ./file.json 
With this command, FileZilla (FTP app) will be able to do changes in the files/directories in the server
sudo chown -R ec2-user:apache ./file.json 

Set Different Volume Levels For Each App on Mac pro

Here is an app url: click here

Download and run Tensorflow on EC2 AWS

With these commands, you'll be able to install tensorflow in a virtual environment and run it:
#---------------------------------------------
#Create a virtual environment:
#---------------------------------------------
python3 -m venv --system-site-packages ./venv
source ./venv/bin/activate  # sh, bash, or zsh
pip install --upgrade pip
#---------------------------------------------
#Install TensorFlow
#---------------------------------------------
pip install --upgrade tensorflow
deactivate
#---------------------------------------------
#run python file out of the venv
#---------------------------------------------
./venv/bin/python file.py 

Get MAC address of rasperrypi for both eth and wlan

With these commands, you'll be able to get the MAC address of eth0 and wlan0 and print them in whatever text format you wish:
IFACE_E=eth0
read MAC_E </sys/class/net/$IFACE_E/address
IFACE_W=wlan0
read MAC_W </sys/class/net/$IFACE_W/address
MACX=${IFACE_E}${MAC_E}${IFACE_W}${MAC_W}
MACX2=$(echo $MACX | sed 's/[:"]//g')
echo $MACX2

Camera does not work on macbook-Pro

Run the following command
sudo killall VDCAssistant

Raspberry Pi: to find the version e.g, 3 or 4

Run the following command
cat /proc/device-tree/model

Raspberry Pi SD Card to img on Mac on macbook-Pro

Find the disk number
diskutil list
run the command with the disk number (for example here is the disk2):
sudo dd if=/dev/rdisk2 of=aaa.img bs=1m 

PHP: To read the csv file without having to deal with new line issue

Here is the code, the csv file will be save in array: $csv
$filename="aaa.csv";
$file = fopen($filename,"r");
while(! feof($file))
  {
  $csv[]=fgetcsv($file);
  }
fclose($file);

Apahce: Increase the HTTP request size (GET).

First, open the file:
sudo vim /etc/httpd/conf/httpd.conf 
Add the limitationm line (by bytes)
LimitRequestBody 10000000000
LimitRequestLine 10000000000
LimitRequestFieldSize 10000000000
Then restart the server
sudo service httpd restart

PHP: Increase the HTTP request size (POST).

First, open the file:
sudo vim /etc/php.ini
Add the limitation size, or put 0 if you dont want to have any limitation.
post_max_size = 0

PHP: file_get_contents doesn't work anymore on AWS EC2.

You've to add ssl verify array:
$arrContextOptions=array("ssl"=>array("verify_peer"=>false,"verify_peer_name"=>false,),);  
Then you could call the file_get_contents
$output = file_get_contents('http://website.com', false, stream_context_create());

Shell: run in the background without saving the log file: nohup.out:

Note first add the follwing line in the sh file:
#!/bin/bash
Then give the following permsion te the sh file:
sudo chmod +x aaa.sh
ex. if aaa.sh was your sh file you want to run then the format will be:
sudo nohup sh aaa.sh >/dev/null 2>&1 &

Shell: Get the size of folder in a path:

ex. if you want to see the size of folders in the home directory:
sudo du -h --max-depth=1 /home/

Shell: Infinite loop to kill a process by name:

ex. tee is a keyword for a command (replace it with any other word)
while true ; do sudo pkill -f tee ; done

Pyhton: Create empty array

Here is the source code:
import numpy as np
np.array([])

EC2: Create swap area

Ref: click here If swapfile already exist
sudo swapoff -a
The swap file is 4 GB (128 MB x 32):
sudo dd if=/dev/zero of=/swapfile bs=128M count=32
Update the read and write permissions
sudo chmod 600 /swapfile
Set up the swap area:
sudo mkswap /swapfile
Make the swap file available
sudo swapon /swapfile
Verify
sudo swapon -s
Start the swap file at boot
sudo vi /etc/fstab
Add the following line at the end
/swapfile swap swap defaults 0 0

Linux: Run script in the background (even after exiting the terminal)

Using nohup command you can run a shell script file as follows:
nohup sh ./cut-all.sh &

Linux: print a size of directory

If the target directory is folder, here is the command:
sudo du -sh ./folderA

Python3: Virtual environment

Create:
python3 -m venv ./venv
Activate:
source ./venv/bin/activate

Python: check if a URL (file, image,page, etc. exist or not without loading the whole file)

import requests

def is_there(path):
    r = requests.head(path)
    return r.status_code == requests.codes.ok

Python/pandas: Mode with a "group by" in pandas

Let's assume you have a pandas data frame df and you want to group by column A and calculate the mode of column B then calculate the mean of B afterward.
import numpy as np
def mode(a):
    u, c = np.unique(a, return_counts=True)
    return u[c.argmax()]
#####
g=pd.DataFrame(df[['A','B']].groupby(['A']).apply(mode) )
g.columns=["B"]
XMODE=g['B'].mean()
Ref: click here

Python/pandas/matplotlib: Fill the area between 2 pandas curves

Let's assume you have a pandas data frame df and you want to fill the area between columns A and B. With a grey color and transparency of 0.5.
plt.fill_between(df.index, df['A'],df['B'], interpolate=True, color='grey', alpha=0.5)
Ref: click here

Linux: copy directory dir1 to another one dir2 without losing the contents in the target directory

rsync -r dir1/ dir2

Python/Pandas: filter a columns of data-frame based on contains a partial of string

Let's assume we have a data-frame A and a column called 'user'. Let's say we want to filter the users that contain 1234
B=A[A['user'].str.contains("1234")]
If we want the once that doesn't contain 1234
B=A[~A['user'].str.contains("1234")]

Linux: go to the end of the command in htop

To go to the beginning
 Ctrl+A 
To go the end
 Ctrl+E 

Linux: protect folder by cookies

Edit the following file:
sudo vim /etc/httpd/conf/httpd.conf
Add the following commandL

RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_COOKIE} !user=admin [NC]
RewriteRule ^ https://example.net/error.php [NC,L]

Then restart the apache server
sudo /etc/init.d/httpd restart

Python/panda: read csv using read_csv via https url

Use the follwoing commandas:
import io
import requests
path="https://example.com/a.csv"
s=requests.get(path).content
cont=io.StringIO(s.decode('utf-8'))
dfx=pd.read_csv(cont,header=None)

AWS/EC2: crontab

To edit the root crontab
sudo vim /etc/crontab

AWS/EC2: certbot & Let's Encrypt certificates

Follow the instruction in the following link: click here
sudo yum update
sudo yum install python3 python3-venv libaugeas0
sudo python3 -m venv /opt/certbot/
sudo /opt/certbot/bin/pip install --upgrade pip
sudo /opt/certbot/bin/pip install certbot certbot-apache
sudo ln -s /opt/certbot/bin/certbot /usr/bin/certbot
Certbot edit the apache configuration automatically:
sudo certbot --apache
Or, just get a certificate:
sudo certbot certonly --apache
Auto renew:
echo "0 0,12 * * * root /opt/certbot/bin/python -c 'import random; import time; time.sleep(random.random() * 3600)' && sudo certbot renew -q" | sudo tee -a /etc/crontab > /dev/null

AWS/EC2: set up an EC2/Apache/PHP/MySQL

Step 1 (set up an EC2 instance)

Go to: https://aws.amazon.com/
Create an account (you could use your Amazon account)
Create an EC2 instance.

Step 2 (access the EC2 instance via SSH)

(1) From the Command line we have to change the permission of the Key to make it private:
chmod 400 ttest.pem
(2) Then we will need to use the following info:
Username: ec2-user
Public DNS: *.*.amazonaws.com [or you could use the IP instead]
Key: test.pem
(3) From the command line make the SSH connection
ssh -i ttest.pem ec2-user@*.*.amazonaws.com

Step 3 (set up Apache/PHP/MySql)

(0) you may update the packages
sudo yum update –y
#sudo yum update
(1) download the PHP package
#sudo yum install -y httpd24 php56 mysql55-server php56-mysqlnd
sudo yum install -y httpd24 php70 mysql56-server php70-mysqlnd
(2) Start the HTTP service
sudo service httpd start
#sudo chkconfig httpd on
#chkconfig --list httpd
(3) Go the folder of www (online)
cd /var/www/
(4) give permission to add/delete and edit for the ec2-user
sudo chown -R ec2-user:apache /var/www

AWS/EC2: Step 4 (setup mysql_secure_installation)

(1) Start MySQL server.
sudo service mysqld start
(2) Run mysql_secure_installation.
sudo mysql_secure_installation
(3) Setup password for example
username: root
Password: 123456
(4) In case you want to stop MySql
sudo service mysqld stop
(5) Auto run when setup server
sudo chkconfig mysqld on

AWS/EC2: Step 5 (Install phpMyAdmin)

sudo yum install php70-mbstring.x86_64 php70-zip.x86_64 -y
sudo service httpd restart
cd /var/www/html
wget https://www.phpmyadmin.net/downloads/phpMyAdmin-latest-all-languages.tar.gz
mkdir phpMyAdmin && tar -xvzf phpMyAdmin-latest-all-languages.tar.gz -C phpMyAdmin --strip-components 1
rm phpMyAdmin-latest-all-languages.tar.gz
sudo service mysqld start
To access mysql:
mysql -u root -p

AWS/Apache: check the url requests

Open config file sudo vim /etc/httpd/conf/httpd.conf Put the following, and change example.com to your domain SetHandler server-status Require host example.net AllowOverride None # Allow open access: Require all granted Then restart apache sudo service httpd restart