Pyenv

加速 从其他镜像站下载源码到 $PYENV_ROOT/.pyenv/cache 目录,然后执行安装 export v=3.11.3 curl -L https://npm.taobao.org/mirrors/python/$v/Python-$v.tar.xz -o ~/.pyenv/cache/Python-$v.tar.xz pyenv install $v pyenv-update 是一个pyenv的插件,用于更新插件和可安装版本列表 $ git clone https://github.com/pyenv/pyenv-update.git $(pyenv root)/plugins/pyenv-update $ pyenv update

May 17, 2023 · 1 min · Peter

sqlite视图

问题 在Centos上遇到sqlite3版本过低的问题, 3.7., 但是django要求版本不低于3.8., 自己编译升级后, 还是无法解决: 原因是python找寻动态库的位置不对 # 下载并解压 [root@linux ~]# wget https://www.sqlite.org/2019/sqlite-autoconf-3270200.tar.gz [root@linux ~]# tar xf sqlite-autoconf-3270200.tar.gz # configure 生成makefile并指定安装路径 [root@linux ~]# cd sqlite-autoconf-3270200 [root@linux sqlite-autoconf-3270200]# ./configure --prefix=/usr/local/ [root@linux sqlite-autoconf-3270200]# make && make install # 备份旧版本 [root@linux ~]# mv /usr/bin/sqlite3 /usr/bin/sqlite3_backupold # 链接新版本 [root@linux ~]# ln -s /usr/local/bin/sqlite3 /usr/bin/sqlite3 # 环境变量设置 [root@linux ~]# vim ~/.bashrc 添加内容如下, LD_LIBRARY_PATH 默认是没有设置的, 如果设置了会优先使用 export LD_LIBRARY_PATH="/usr/local/lib" json1 参考:howto_get_going_with_sqlite_json1 使用版本3.7.17,最低支持版本3.9.0 尝试编译json.so 去3.7.17中.load json.so 结论:3.7.17 可以加载,但是运行报错segmentation fault,应该还是有代码不兼容,直接在3.9.0上可加载、可运行 下载: ...

April 26, 2023 · 3 min · Peter

Fabric

Fabric学习 [toc] 总结 fabric3 移除了很多实用的功能和装饰器,变得更精简了,另外fab这个cli入口提供的参数能力也变弱了,目前个人实用的方式是typer+fabric3+makefile: typer 作为参数解析入口 makefile 固定一些操作逻辑,可以使用它的并发操作,而不是ThreadingGroup Link Welcome to Fabric! Welcome to Invoke’s documentation! 使用 Fabric 自动化部署 What is Fabric python的一个high level的库,通过ssh执行shell命令。 基于以下两个包构建的: Invoke,subprocess command execution and command-line features Paramiko,SSH protocol implementation 私钥配置 配置ssh的authorized_keys,将本机的id_rsa.pub信息复制到远程的authorized_keys文件中 简要用法 pty(prompt by hand),如果sudo有密码,那么用pty来手动输入 >>> from fabric import Connection >>> c = Connection('192.168.9.139') >>> c.run('sudo useradd mydbuser', pty=True) [sudo] password: <Result cmd='sudo useradd mydbuser' exited=0> >>> c.run('id -u mydbuser') 利用Invoke的自动回应来输入sudo密码,注意pattern的正确匹配,以及repsonese的结尾符号 >>> from invoke import Responder >>> from fabric import Connection >>> c = Connection('host') >>> sudopass = Responder( ... pattern=r'\[sudo\] password:', ... response='mypassword\n', ... ) >>> c.run('sudo whoami', pty=True, watchers=[sudopass]) [sudo] password: root <Result cmd='sudo whoami' exited=0> watchers虽然提供了自动填充密码的功能,但是代码中多次引用这种配置比较麻烦,所以考虑将sudopass的内容放入配置里面,让Connection.sudo来处理剩下的工作。 >>> import getpass >>> from fabric import Connection, Config >>> sudo_pass = getpass.getpass("What's your sudo password?") What's your sudo password? >>> config = Config(overrides={'sudo': {'password': sudo_pass}}) >>> c = Connection('db1', config=config) >>> c.sudo('whoami', hide='stderr') root <Result cmd="...whoami" exited=0> >>> c.sudo('useradd mydbuser') <Result cmd="...useradd mydbuser" exited=0> >>> c.run('id -u mydbuser') 1001 <Result cmd='id -u mydbuser' exited=0> 文件传输,transfer files,利用Connection.put和Connection.get。在使用时发现如果像下面这样定义remote的话,会报错OSError,要带上远程的文件重命名就不会报错。 >>> from fabric import Connection >>> result = Connection('web1').put('myfiles.tgz', remote='/opt/mydata/') >>> print("Uploaded {0.local} to {0.remote}".format(result)) Uploaded /local/myfiles.tgz to /opt/mydata/ 之前都是单行的操作,实际多半不是这样,多个同时执行,如上传、解压,路径、文件名可以作为参数传入,以便复用函数 from fabric import Connection c = Connection('web1') c.put('myfiles.tgz', '/opt/mydata') c.run('tar -C /opt/mydata -xzvf /opt/mydata/myfiles.tgz') def upload_and_unpack(c): c.put('myfiles.tgz', '/opt/mydata') c.run('tar -C /opt/mydata -xzvf /opt/mydata/myfiles.tgz') 多服务器的情况,这个单列出来 最直观的方法是遍历一个connection列表 >>> from fabric import Connection >>> for host in ('web1', 'web2', 'mac1'): >>> result = Connection(host).run('uname -s') ... print("{}: {}".format(host, result.stdout.strip())) ... ... web1: Linux web2: Linux mac1: Darwin fabric提供了一个更好的解决办法,把所有的对象放到一个Group里面。 Group和SerialGroup(Subclass of Group which executes in simple, serial fashion. New in version 2.0.) 返回的结果看,Connection返回单个的Result对象,Group返回GroupResult对象,dict-like In [18]: results = Group('peter@192.168.9.218', 'peter@192.168.9.208').run('uname -s') Linux Linux In [19]: for conn, result in results.items(): ...: print("{0.host}: {1.stdout}".format(conn, result)) ...: 192.168.9.218: Linux 192.168.9.208: Linux Group对象来执行的操作会对所有服务器生效,但是这里官方教程的pool.put会得到一个AttributeError: 'SerialGroup' object has no attribute 'put'错误,已在github提交issue from fabric import SerialGroup as Group pool = Group('web1', 'web2', 'web3') pool.put('myfiles.tgz', '/opt/mydata') pool.run('tar -C /opt/mydata -xzvf /opt/mydata/myfiles.tgz') fab cli tool 封装了Invoke的CLI功能,能够很快的在多个机器上运行tasks。 ...

March 30, 2023 · 2 min · Peter

pip dependency

查看依赖 工具 pipdeptree 支持正向、反向查看依赖 多种格式输出,安装了graphviz还可以输出图形 [root@10-113-57-176 /] pip install pipdeptree [root@10-113-57-176 /] pipdeptree --help usage: pipdeptree [-h] [-v] [-f] [--python PYTHON] [-a] [-l] [-u] [-w [{silence,suppress,fail}]] [-r] [-p PACKAGES] [-e PACKAGES] [-j] [--json-tree] [--graph-output OUTPUT_FORMAT] Dependency tree of the installed python packages optional arguments: -h, --help show this help message and exit -v, --version show program's version number and exit -f, --freeze Print names so as to write freeze files --python PYTHON Python to use to look for packages in it (default: where installed) -a, --all list all deps at top level -l, --local-only If in a virtualenv that has global access do not show globally installed packages -u, --user-only Only show installations in the user site dir -w [{silence,suppress,fail}], --warn [{silence,suppress,fail}] Warning control. "suppress" will show warnings but return 0 whether or not they are present. "silence" will not show warnings at all and always return 0. "fail" will show warnings and return 1 if any are present. The default is "suppress". -r, --reverse Shows the dependency tree in the reverse fashion ie. the sub- dependencies are listed with the list of packages that need them under them. -p PACKAGES, --packages PACKAGES Comma separated list of select packages to show in the output. If set, --all will be ignored. -e PACKAGES, --exclude PACKAGES Comma separated list of select packages to exclude from the output. If set, --all will be ignored. -j, --json Display dependency tree as json. This will yield "raw" output that may be used by external tools. This option overrides all other options. --json-tree Display dependency tree as json which is nested the same way as the plain text output printed by default. This option overrides all other options (except --json). --graph-output OUTPUT_FORMAT Print a dependency graph in the specified output format. Available are all formats supported by GraphViz, e.g.: dot, jpeg, pdf, png, svg

February 10, 2023 · 2 min · Peter

Daylight Saving Time

参考 https://www.cnblogs.com/zihanxing/articles/6224263.html 验证 zdump -v /usr/share/zoneinfo/America/Los_Angeles | grep 2022 ln -sf /usr/share/zoneinfo/America/Los_Angeles /etc/localtime service ntpd stop date -s "2022-03-13 01:59:40" for i in `seq 1 1000`;do date;python -c 'import time;print(int(time.time()))';python -c 'import datetime;print(datetime.datetime.now())';sleep 1;done

January 4, 2023 · 1 min · Peter