Shaokun Xie's Notebook
Fix zip file names using unzip
Dec 2, 2024
Double-check the exact name of the encoding, as to not misspell it: https://www.iana.org/assignments/character-sets/character-sets.xhtml
The run
$ unzip -O <encoding> <filename> -d <target_dir> or
$ unzip -I <encoding> <filename> -d <target_dir> choosing between -O or -I according to instructions here:
$ unzip -h UnZip 6.00 of 20 April 2009, by Debian. Original by Info-ZIP. ... -O CHARSET specify a character encoding for DOS, Windows and OS/2 archives -I CHARSET specify a character encoding for UNIX and other archives .
Support Synaptics Fingerprint Reader 06cb:00be
Apr 28, 2024
A bit background 🔗Lenovo did’t release the official driver and libfprint could not support it issue #296 . However, luckily, there is a nice GitHub repo that provides a viable solution.
Let’s go 🔗Here I used Ubuntu, but other distros should be more or less the same while the package manager can be different, the naming of the deps might vary a little.
Clone the project 🔗git clone https://github.com/Popax21/synaTudor.git Install dependencies 🔗# the tools if you don't already have $ sudo apt-get install -y meson ninja cmake # build deps $ sudo apt-get install -y libgusb-dev libcap-dev libseccomp-dev libfprint-2-tod-dev libjson-glib-dev innoextract # fprint $ sudo apt-get install -y fprintd libpam-fprintd Build and install 🔗cd synaTudor meson build cd build ninja sudo ninja install Voilà , you should have the fingerprint option enabled in your desktop environment at this moment, or you can use fprint to manage manually:
PostgreSQL Replication
Nov 3, 2023
Concepts & Terms 🔗WAL 🔗Write-Ahead Log (WAL) is a record of the changes made to the data. It ensures that when there is a crash in the system or loss of connection, the database can be recovered.
WAL Level 🔗WAL level determines how much information is written to the WAL, it corresponds to the wal_level option in the postgresql.conf file. The default value is replica, which writes enough data to support WAL archiving and replication, including running read-only queries on a standby server.
Exceptions logging in python
Nov 18, 2022
There are several ways of logging exception information in Python:
import sys, logging l = logging.getLogger('test') l.addHandler(logging.StreamHandler(sys.stdout)) def foo(): try: raise KeyError("haha, an error") except Exception as e: l.error("%ssep %s", "hello", e, exc_info=True) print("--------") l.error("%ssep %s", "hello", e) print("--------") l.exception(e) print("--------") l.exception("", exc_info=e) print("--------") l.exception(e, exc_info=1) print("--------") print(str(e)) print("--------") print(repr(e)) The output would be:
hello sep 'haha, an error' Traceback (most recent call last): File "<ipython-input-1-3354851c0b12>", line 7, in foo raise KeyError("haha, an error") KeyError: 'haha, an error' -------- hello sep 'haha, an error' -------- 'haha, an error' Traceback (most recent call last): File "<ipython-input-1-3354851c0b12>", line 7, in foo raise KeyError("haha, an error") KeyError: 'haha, an error' -------- Traceback (most recent call last): File "<ipython-input-1-3354851c0b12>", line 7, in foo raise KeyError("haha, an error") KeyError: 'haha, an error' -------- 'haha, an error' Traceback (most recent call last): File "<ipython-input-1-3354851c0b12>", line 7, in foo raise KeyError("haha, an error") KeyError: 'haha, an error' -------- 'haha, an error' -------- KeyError('haha, an error') logger.
Some notes about NLP
Nov 2, 2021
stemming and lemmatization 🔗 lemmatization: like stemming but gives human-understandable word bag-of-words 🔗 purpose: ML accept only vectors of same length after stemming/lemmatization ignoring case etc, build vectors for each sentence, of which each variable corresponds to the score of a word in the vacabulary words hashing: use a hash function to map words to a hash space(vector) TF(Term Frequency)-IDF 🔗 TF = repetition of a word in a document / number of words in a document IDF = log(number of all document / number of document containing the word) The final result is a table where for each word, its score is given by the product of the score in IF and the socre in IDF Word2Vec 🔗 word embedding represent a word by a vector of features uses a neural network model to learn word associations from a large corpus of text and produce word embeddings
Python MRO
Mar 10, 2021
Like 2 years ago I firstly learned about the python3 MRO(method resolution order), I though I pretty much got the hang of it, until I met this:
class X:pass class Y: pass class Z:pass class A(X,Y):pass class B(Y,Z):pass class M(B,A,Z):pass Apprently the search order should be
M B Y Z A X Y Z
thus the MRO is supposed to be
M B Y Z A X
Problem-solving with State Graph
Mar 10, 2021
Typology of three types of problem 🔗1. combinatoire 🔗definition
trouver dans un ensemble X donne les elements x satisfaisant un ensemble de contraintes donnees K(x)
2. changement 🔗3. décompositon en sou-problèmes 🔗General problem solver 🔗The idea is to finda a way to reduce the difference between the initial state and the desired state.
transformation
Build a list of differences between object A and object B
Reduction
Modèles Et Algorithmes Parallèles
Mar 4, 2021
Classification de Flynn 🔗 SISD mono-core PC SIMD some multi-dimensional data, image/matrix etc. MISD not exist MIMD mulitple cores, paralle calculation shared memory distributed memory Matrice to measure perfomance 🔗Definition 🔗T1: The time for one core sequential calculation. Tn: The time for parallel calculation. N signifies the number of processors. speed_up = T1/Tn O: The overhead, the cost of communication between processors, synchronization, etc.. Fixed-sized model (Loi d’Amdahl) 🔗The proportion f part is fixed, which means it is independent from n.