[Python] Multiprocessing example

Python / 2021 年 12 月 17 日
import multiprocessing as mp

def task(x):
    # Do something
    return x

if __name__ == '__main__':
    with mp.Pool(mp.cpu_count()) as pool:
        results = []
        # Using apply_async
        for i in range(cpu_count):
            result = pool.apply_async(task, args=(i,))
            results.append(result)
        for result in results:
            print(result.get())

        # Using map_async
        result = pool.map_async(task, range(cpu_count))
        print(result.get())

[Python] unittest template for python script

Python / 2021 年 12 月 17 日
 
import unittest
import subprocess

def execute_command(command):
    """Execute a shell command and return its printed value."""
    return subprocess.run(command, stdout=subprocess.PIPE).stdout.decode('utf-8').strip()

class TestTemplate(unittest.TestCase):
    def test_case1(self):
        result = 1
        expected_result = 1
        self.assertEqual(result, expected_result)

if __name__ == '__main__':
    unittest.main(verbosity=2)

[Linux] Screen 常用指令

Linux / 2019 年 8 月 11 日

Start a new session

screen
or
screen -S session_name

List session

screen -ls

Resume session

screen -r session_name

Detach session

screen -d session_name

Shortcuts

Ctrl+a c Create a new window (with shell)
Ctrl+a " List all window
Ctrl+a 0 Switch to window 0 (by number )
Ctrl+a A Rename the current window
Ctrl+a S Split current region horizontally into two regions
Ctrl+a | Split current region vertically into two regions
Ctrl+a tab Switch the input focus to the next region
Ctrl+a Ctrl+a Toggle between the current and previous region
Ctrl+a Q Close all regions but the current one
Ctrl+a X Close the current region
Ctrl+a k Kill the current window
Ctrl+a M Show window monitor

[Docker] 常用指令

Docker / 2022 年 12 月 19 日

Image

  • List all stored images
docker image ls
  • Pull an image
docker pull xxx
  • Build an image
$ docker build -t imagename:2.0 .
  • Delete specific images
docker rmi -f $(docker images -q myname)
# or
docker rmi -f image-name
  • Delete all images
docker rmi -f $(docker images -aq)

Container

  • Show all containers
docker ps -a
  • Run/stop a container
docker run -it ubuntu bash
# -d: Run in detached mode
# -v: HOST_PATH:CONTAINER_PATH
# --restart=always

# Stop a specific container
docker stop container_name

# Stop all containers
docker kill $(docker ps -q)
  • Enter/exit a container
# Enter an exited container
docker start myname
docker exec -it myname bash

# Detach from a container
Ctrl-p + Ctrl-q inside the container shell
# Re-attach into an up container shell
docker attach myname
  • Delete a specific container
docker rm -f container_name
  • Delete all containers
docker rm -f $(docker ps -aq)
  • Mirror ports
# HOST_PORT:CONTAINER_PORT
docker run -p 8000:8000 myimage
  • Copy file from/to container
docker cp test.txt mycontainer:/app/test.txt

Volume

  • List all volumes
docker volume ls
  • Mount a volume from another container
--volumes-from container_name

Networking

  • Link containers (legacy)
--link container_name (legacy)
# Then you can ping container_name
# It’s better to create a network and add the containers to the network as below
  • Create a network
docker network create network-name
# Creates a bridge network by default
  • network mode (–net)
    • none: No network connection
    • container: Mirror a specific container
    • host: Mirror the host
    • bridge: Default. Like NAT
    • custom created network: Overlay or bridge

Tag

  • Tag an image for pushing
docker tag image-name DockerHubAcc/image-name

Save/Export and Load/Import

  • save (for image)
docker save myimage > myimage.tar

# Using gzip for compression
docker save myimage:tag | gzip > myimage_tag.tar.gz
  • load (for image)
docker load < myimage.tar
  • export (for container)
docker export mycontainer > mycontainer.tar
  • import (for container)
docker import mycontainer.tar image-name

Logging/Stats

  • Show log for a container
docker logs container_name
# Use -f to watch the logs
  • Show HW stats
docker stats

Cleanup

  • Delete all stopped containers
docker container prune
  • Clear all space used by images, containers and volumes
docker system prune --volumes  # Only remove dangling images
# -a: Remove all images with no container associated

Dockerfile

  • FROM
  • RUN

Use && and \ to chain commands to avoid too many intermediate images get created.

  • ADD/COPY

ADD can auto extract .tar.gz and handle remote URL, otherwise use COPY.

  • ENV
  • VOLUME

Declare auto created volumes using container paths

  • EXPOSE

Only for declaration purpose. docker run -P takes a random port to map to this exposed port.

  • CMD/ENTRYPOINT

Most likely the same if used alone. If ENTRYPOINT is used, commands are appended as its arguments (either specified in command line or using CMD. CMD will serve as a default argument in this case)

Docker-compose

  • Pull all related images
docker-compose pull
  • Create/delete all objects
# Create all stuffs
docker-compose up -d

# Delete all stuffs
docker-compose down
  • Start/stop the containers
# Start the containers
docker-compose start

# Stop the containers
docker-compose stop
  • Show information
docker-compose logs
docker-compose ps

[Linux] nmap常用指令

Linux / 2021 年 12 月 17 日
  • 掃描是否alive
nmap -sP target_ip_range
  • 掃描開啟的port以及軟體版本
nmap -sV target_ip
  • 掃描作業系統版本
nmap -O target_ip
  • 使用NSE預設腳本
nmap -sC target_ip

[Git] tag相關

Git / 2021 年 12 月 17 日
  • 列出所有tag
git tag -l
  • 新增tag
git tag -a v1.0 -m "my version message"
  • 上傳tag資訊到遠端
git push origin --tags
  • 刪除遠端tag “v1.0”
git push origin :refs/tags/v1.0

[Python] Send an email

Python / 2021 年 12 月 17 日
import smtplib

# user settings
email_addr = 'your@email'
email_pass = 'yourpassword' # you might need to use an application password
msg = "your message"

# send simple email
server = smtplib.SMTP('smtp.gmail.com', 587)
server.starttls()
server.login(email_addr, email_pass)
server.sendmail(email_addr, email_addr, msg)
server.quit()