detectron2 window 快速安装

参考:
https://medium.com/@dgmaxime/how-to-easily-install-detectron2-on-windows-10-39186139101c
==1==
https://docs.conda.io/en/latest/miniconda.html
安装conda,设置镜像地址
pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/
pip config set global.trusted-host mirrors.aliyun.com
==2==
安装第三方模块
Install Cuda
Install Torch
pip install opencv-python
==3==
安装VS2019(VS2017)
==4==
https://github.com/DGMaxime/detectron2-windows.git
cd D:\apdata\test\detectron2-windows
“C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars64.bat”
set DISTUTILS_USE_SDK=1
python setup.py build develop
pip install -e .

==5==
https://pypi.org/project/pycocotools-windows/
pip install opencv-python
pip install numpy scipy pandas matplotlib scikit-learn
pip install cython
pip install pycocotools-windows
pip install Cython
pip install Pillow
pip install yacs
pip install termcolor
pip install cityscapesScripts
pip install tabulate
pip install tqdm
pip install lvis
pip install psutil
pip install fvcore
pip install pycocotools
caffe2 需要编译,
https://download.pytorch.org/whl/torch_stable.html
# CUDA 10.2
pip install torch==1.6.0 torchvision==0.7.0
# CUDA 10.1
pip install torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html
# CUDA 9.2
pip install torch==1.6.0+cu92 torchvision==0.7.0+cu92 -f https://download.pytorch.org/whl/torch_stable.html
# CPU only
pip install torch==1.6.0+cpu torchvision==0.7.0+cpu -f https://download.pytorch.org/whl/torch_stable.html
pip install torch==1.5.1+cu92 torchvision==0.6.1+cu92 -f https://download.pytorch.org/whl/torch_stable.html
pip install cv2 PIL pycocotools yacs termcolor cityscapesscripts tabulate tqdm scipy lvis psutil pkg_resources caffe2 onnx
“C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Auxiliary\Build\vcvars64.bat”
python setup.py build develop
cd D:\apdata\test\detectron2-windows
set DISTUTILS_USE_SDK=1
python demo.py –config-file D:/apdata/test/detectron2-windows/configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml –webcam –output outputs –opts MODEL.WEIGHTS ‘D:/apdata/test/detectron2-windows/demo/R-50.pkl’
python demo.py –config-file D:/apdata/test/detectron2-windows/configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml –webcam –opts MODEL.WEIGHTS ‘D:/apdata/test/detectron2-windows/demo/R-50.pkl’
python C:/test/detectron2-windows/demo/demo.py –config-file C:/test/detectron2-windows/configs/COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml –webcam –opts MODEL.WEIGHTS ‘C:/test/detectron2-windows/demo/R-50.pkl’
https://pypi.org/project/pycocotools-windows/

自动加载VHD脚本


@echo off
@chcp 65001
@title %~n0
@cd /d %~dp0
@echo %~dp0
@echo %~n0
@echo %~dp0%~n0
@echo.


@set vhd_file=%~dp0%~n0
@set vhd_script=%~dp0%~n0.txt
@echo vhd_file,%vhd_file%
@echo vhd_script,%vhd_script%


if not exist "%vhd_file%" goto go_nofile
echo 运行正常程序 start
echo select vdisk file="%vhd_file%"  > %vhd_script%
echo attach vdisk  >> %vhd_script%
call diskpart.exe /s %vhd_script%
del %vhd_script%
echo 运行正常程序 end


:go_exit
@exit

:go_nofile
@echo go_nofile
@pause 

MySQL,查询所有库,表,及列,信息,及常用配置

MySQL,查询所有库,表,及列,信息

SELECT
C.TABLE_SCHEMA AS '库名',
T.TABLE_NAME AS '表名',
T.TABLE_COMMENT AS '表注释',
C.COLUMN_NAME AS '列名',
C.COLUMN_COMMENT AS '列注释',
C.ORDINAL_POSITION AS '列的排列顺序',
C.COLUMN_DEFAULT AS '默认值',
C.IS_NULLABLE AS '是否为空',
C.DATA_TYPE AS '数据类型',
C.CHARACTER_MAXIMUM_LENGTH AS '字符最大长度',
C.NUMERIC_PRECISION AS '数值精度(最大位数)',
C.NUMERIC_SCALE AS '小数精度',
C.COLUMN_TYPE AS 列类型,
C.COLUMN_KEY 'KEY',
C.EXTRA AS '额外说明'
FROM
information_schema.`TABLES` T
LEFT JOIN information_schema.`COLUMNS` C ON T.TABLE_NAME = C.TABLE_NAME
AND T.TABLE_SCHEMA = C.TABLE_SCHEMA
ORDER BY C.TABLE_NAME, C.ORDINAL_POSITION;

MySQL 升级支持时间表

 

===========================
MySQL.8.4-升级变化
===========================

MySQL 8.4.0 LTS 已经发布 ,作为发版模型变更后的第一个长期支持版本,注定要承担未来生产环境的重任,那么这个版本都有哪些新特性、变更,接下来少安将带大家一起来 get 新知识点。

INFORMATION_SCHEMA 变更

表 INFORMATION_SCHEMA.TABLESPACES 从 MySQL 8.0.22 开始被废弃,现已移除。

  • 8.4.0
mysql> select version()\G
*************************** 1. row ***************************
version(): 8.4.0-ShawnYan-debug
1 row in set (0.00 sec)

mysql> desc INFORMATION_SCHEMA.TABLESPACES;
ERROR 1109 (42S02): Unknown table 'TABLESPACES' in information_schema
  • 8.3.0
mysql> select version()\G
*************************** 1. row ***************************
version(): 8.3.0
1 row in set (0.01 sec)

mysql> desc INFORMATION_SCHEMA.TABLESPACES;
+--------------------+-----------------+------+-----+---------+-------+
| Field              | Type            | Null | Key | Default | Extra |
+--------------------+-----------------+------+-----+---------+-------+
| TABLESPACE_NAME    | varchar(64)     | NO   |     |         |       |
| ENGINE             | varchar(64)     | NO   |     |         |       |
| TABLESPACE_TYPE    | varchar(64)     | YES  |     |         |       |
| LOGFILE_GROUP_NAME | varchar(64)     | YES  |     |         |       |
| EXTENT_SIZE        | bigint unsigned | YES  |     |         |       |
| AUTOEXTEND_SIZE    | bigint unsigned | YES  |     |         |       |
| MAXIMUM_SIZE       | bigint unsigned | YES  |     |         |       |
| NODEGROUP_ID       | bigint unsigned | YES  |     |         |       |
| TABLESPACE_COMMENT | varchar(2048)   | YES  |     |         |       |
+--------------------+-----------------+------+-----+---------+-------+
9 rows in set (0.20 sec)

对于 InnoDB 表,Information Schema 中的表 INNODB_TABLESPACES 和 INNODB_DATAFILES 来提供表空间元信息。

mysql> table INFORMATION_SCHEMA.INNODB_TABLESPACES;
+------------+------------------+-------+----------------------+-----------+---------------+------------+---------------+-----------+----------------+-----------------+----------------+---------------+------------+--------+
| SPACE      | NAME             | FLAG  | ROW_FORMAT           | PAGE_SIZE | ZIP_PAGE_SIZE | SPACE_TYPE | FS_BLOCK_SIZE | FILE_SIZE | ALLOCATED_SIZE | AUTOEXTEND_SIZE | SERVER_VERSION | SPACE_VERSION | ENCRYPTION | STATE  |
+------------+------------------+-------+----------------------+-----------+---------------+------------+---------------+-----------+----------------+-----------------+----------------+---------------+------------+--------+
| 4294967294 | mysql            | 18432 | Any                  |     16384 |             0 | General    |          4096 |  26214400 |       26214400 |               0 | 8.4.0          |             1 | N          | normal |
| 4294967293 | innodb_temporary |  4096 | Compact or Redundant |     16384 |             0 | System     |          4096 |  12582912 |       12582912 |               0 | 8.4.0          |             1 | N          | normal |
| 4294967279 | innodb_undo_001  |     0 | Undo                 |     16384 |             0 | Undo       |          4096 |  16777216 |       16777216 |               0 | 8.4.0          |             1 | N          | active |
| 4294967278 | innodb_undo_002  |     0 | Undo                 |     16384 |             0 | Undo       |          4096 |  16777216 |       16777216 |               0 | 8.4.0          |             1 | N          | active |
|          1 | sys/sys_config   | 16417 | Dynamic              |     16384 |             0 | Single     |          4096 |    114688 |         114688 |               0 | 8.4.0          |             1 | N          | normal |
|          2 | yandb/t          | 16417 | Dynamic              |     16384 |             0 | Single     |          4096 |    114688 |         114688 |               0 | 8.4.0          |             1 | N          | normal |
+------------+------------------+-------+----------------------+-----------+---------------+------------+---------------+-----------+----------------+-----------------+----------------+---------------+------------+--------+
6 rows in set (0.00 sec)

mysql> table INFORMATION_SCHEMA.INNODB_DATAFILES;
+------------------------+----------------------+
| SPACE                  | PATH                 |
+------------------------+----------------------+
| 0x30                   | ibdata1              |
| 0x34323934393637323739 | ./undo_001           |
| 0x34323934393637323738 | ./undo_002           |
| 0x31                   | ./sys/sys_config.ibd |
| 0x32                   | ./yandb/t.ibd        |
+------------------------+----------------------+
5 rows in set (0.00 sec)

知识扩展

表 INNODB_TABLESPACES 和 INNODB_DATAFILES 自 MySQL 8.0.3 发生过表名变更,如果从 MySQL 5.7 升级上来,需要注意。

Old NameNew Name
INNODB_SYS_TABLESPACESINNODB_TABLESPACES
INNODB_SYS_DATAFILESINNODB_DATAFILES

权限变更

1. FLUSH_PRIVILEGES

MySQL 8.4.0 LTS 添加了特定于 FLUSH PRIVILEGES 语句的使用的权限。与现有的 RELOAD 权限不同,新的 FLUSH_PRIVILEGES 权限仅适用于 FLUSH PRIVILEGES 语句。此权限的范围是全局的,并且适用于用户和角色。

示例:

  1. 创建带有 FLUSH_PRIVILEGES 权限的用户。
mysql> create user u1;
Query OK, 0 rows affected (0.02 sec)

mysql> grant FLUSH_PRIVILEGES on *.* to u1;
Query OK, 0 rows affected (0.01 sec)

mysql> show grants for u1;
+-------------------------------------------+
| Grants for u1@%                           |
+-------------------------------------------+
| GRANT USAGE ON *.* TO `u1`@`%`            |
| GRANT FLUSH_PRIVILEGES ON *.* TO `u1`@`%` |
+-------------------------------------------+
2 rows in set (0.00 sec)
  1. 使用 u1 用户连接数据库。
mysql> select current_user()\G
*************************** 1. row ***************************
current_user(): u1@%
1 row in set (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.04 sec)
  1. 回收权限后再次使用 u1 刷新权限,执行报错,提示需要 RELOAD 或 FLUSH_PRIVILEGES 权限。
mysql> flush privileges;
ERROR 1227 (42000): Access denied; you need (at least one of) the RELOAD or FLUSH_PRIVILEGES privilege(s) for this operation

同时,RELOAD 权限将继续提供向后兼容性; 具有此权限的用户在升级后仍然可以执行 FLUSH PRIVILEGES 语句。作为升级到 MySQL 8.4 版本的一部分,将执行检查以查看是否有任何用户具有 FLUSH_PRIVILEGES 权限; 如果没有,则具有 RELOAD 权限的用户也会自动分配新权限。

2. OPTIMIZE_LOCAL_TABLE

MySQL 8.4.0 LTS 添加了新的 OPTIMIZE_LOCAL_TABLE 权限。

用户必须具有此权限才能执行 OPTIMIZE LOCAL TABLEOPTIMIZE NO_WRITE_TO_BINLOG TABLE 语句。

从以前的版本升级时,已具有 SYSTEM_USER 权限的用户将被自动授予 OPTIMIZE_LOCAL_TABLE 权限。

mysql> show grants;
+-----------------------------------------------+
| Grants for u1@%                               |
+-----------------------------------------------+
| GRANT USAGE ON *.* TO `u1`@`%`                |
| GRANT OPTIMIZE_LOCAL_TABLE ON *.* TO `u1`@`%` |
| GRANT REFERENCES ON `yandb`.* TO `u1`@`%`     |
+-----------------------------------------------+
3 rows in set (0.00 sec)

mysql> OPTIMIZE LOCAL TABLE t;
+---------+----------+----------+-------------------------------------------------------------------+
| Table   | Op       | Msg_type | Msg_text                                                          |
+---------+----------+----------+-------------------------------------------------------------------+
| yandb.t | optimize | note     | Table does not support optimize, doing recreate + analyze instead |
| yandb.t | optimize | status   | OK                                                                |
+---------+----------+----------+-------------------------------------------------------------------+
2 rows in set (0.20 sec)

mysql> OPTIMIZE NO_WRITE_TO_BINLOG TABLE t;
+---------+----------+----------+-------------------------------------------------------------------+
| Table   | Op       | Msg_type | Msg_text                                                          |
+---------+----------+----------+-------------------------------------------------------------------+
| yandb.t | optimize | note     | Table does not support optimize, doing recreate + analyze instead |
| yandb.t | optimize | status   | OK                                                                |
+---------+----------+----------+-------------------------------------------------------------------+
2 rows in set (0.19 sec)

关键字

从 MySQL 8.0 到 8.4,增加了几个关键字,标记为(R)的是保留关键字:

AUTO, BERNOULLI, GTIDS, LOG, MANUAL (R), PARALLEL (R), PARSE_TREE, QUALIFY (R), S3, and TABLESAMPLE (R).

在准备升级数据库版本时,需要确认库、表有无使用到保留关键字,如果有建议先改名。

  • 确认 SCHEMA 名称没有使用保留关键字
with t as ( 
select 'manual' as rk 
union select 'parallel' 
union select 'qualify' 
union select 'tablesample' 
)
select SCHEMA_NAME from information_schema.SCHEMATA
where lower(SCHEMA_NAME) in (select rk from t);

示例输出:

+-------------+
| SCHEMA_NAME |
+-------------+
| manual      |
+-------------+
1 row in set (0.00 sec)
  • 确认 TABLE 名称没有使用保留关键字
with t as ( 
select 'manual' as rk 
union select 'parallel' 
union select 'qualify' 
union select 'tablesample' 
)
select table_schema, table_name, table_type 
from information_schema.tables 
where lower(table_schema) in (select rk from t) or lower(table_name) in (select rk from t);

示例输出:

+--------------+------------+------------+
| TABLE_SCHEMA | TABLE_NAME | TABLE_TYPE |
+--------------+------------+------------+
| yandb        | manual     | BASE TABLE |
| yandb        | manuaL     | VIEW       |
+--------------+------------+------------+
2 rows in set (0.00 sec)

使用group_concat()去除重复数据语句

SELECT `属性字段` GROUP_CONCAT(DISTINCT `属性名称`) AS `属性名称集合`
FROM 数据表
GROUP BY `属性字段`

 

客户端

  1. mysql_upgrade 移除

在之前的文章 [MySQL 升级:使用 mysql_upgrade 注意事项(再见 mysql_upgrade)]() 中介绍过 mysql_upgrade 的变化。

MySQL 8.4.0 移除了该客户端。

  1. mysqlpump 移除

移除了已弃用的 mysqlpump 客户端,及其关联的 lz4_decompress 和 zlib_decompress 辅助工具。

请使用 mysqldump 或 MySQL Shell 的转储程式。

  1. mysqldump 新选项

mysqldump 新增选项 --output-as-version,用于确定与复制和时间相关的语句所使用的术语级别,从而创建兼容老版本的备份。

该选项有3个可选值,SERVER,BEFORE_8_0_23,BEFORE_8_2_0,默认为 SERVER

BEFORE_8_2_0 将输出已弃用的事件 DISABLE ON SLAVE 术语

BEFORE_8_0_23 将使用已弃用的 SLAVE/CHANGE MASTER 术语来表示复制命令

  1. 客户端注释保留

这里还有一个变更点值得注意。在之前的版本中,MySQL 客户端默认剥离注释。从 MySQL 8.1 开始,默认情况已更改为保留此类注释。

JDBC配置

mysql官方提供的JDBC Driver,其JAVA类名为:com.mysql.jdbc.Driver, 新版为com.mysql.cj.jdbc.Driver
格式说明

jdbc:mysql://[host:port],[host:port].../[database][?参数名1][=参数值1][&参数名2][=参数值2]...

参考

jdbc:mysql://localhost:3306/test?user=root&password=&useUnicode=true&characterEncoding=gbk&autoReconnect=true&failOverReadOnly=false
jdbc:mysql://localhost:3306/test?useUnicode=true&characterEncoding=utf-8&useSSL=false&allowPublicKeyRetrieval=true&nullCatalogMeansCurrent=true&useInformationSchema=true
参数名称参数说明缺省值最低版本要求
user数据库用户名(用于连接数据库)所有版本
password用户密码(用于连接数据库)所有版本
useUnicode是否使用Unicode字符集,如果参数characterEncoding设置为gb2312或gbk,本参数值必须设置为truefalse1.1g
characterEncoding当useUnicode设置为true时,指定字符编码。比如可设置为gb2312或gbkfalse1.1g
autoReconnect当数据库连接异常中断时,是否自动重新连接?false1.1
autoReconnectForPools是否使用针对数据库连接池的重连策略false3.1.3
failOverReadOnly自动重连成功后,连接是否设置为只读?true3.0.12
maxReconnectsautoReconnect设置为true时,重试连接的次数31.1
initialTimeoutautoReconnect设置为true时,两次重连之间的时间间隔,单位:秒21.1
connectTimeout和数据库服务器建立socket连接时的超时,单位:毫秒。 0表示永不超时,适用于JDK 1.4及更高版本03.0.1
socketTimeoutsocket操作(读写)超时,单位:毫秒。 0表示永不超时03.0.1

MySQL8-清理日志

-- 清理日志
-- show binary LOGS;
-- purge binary logs to 'binlog.000142';
set global binlog_expire_logs_seconds = 60*60*24*5;
-- 参数过期
-- set global binlog_expire_logs_days = 5;
-- 432000, 5Day,2592000,30day
show variables like '%log%';
-- expire_logs_days

====
设置MySQL超时问题

 set global INTERACTIVE_TIMEOUT=1000*35;
 set global wait_timeout=1000*35;
 show global  variables where Variable_name in ('interactive_timeout', 'wait_timeout');