humanleft
b7c4653f95
test: 修复 mqy20260511 合并后的测试编译和运行时失败
- MqttFaultConsumerTest: insertAlertData 9→8参数, updateAlertData 7→6参数
- SysRealtimeServiceTest: insertTables/updateTables 参数数量同步方法签名
- MqttChargeStationConsumerTest: pe_iot 前缀改为 pe_ems
- TDengineServiceTest: 11个 getValueType 测试同步 (String, Object) 新签名
2週間前
lenovo
570158c3a4
修改问题:
1.时间改timestamp类型
2.pe_iot 改 pe_ems
3.改hash方法
4.修改tdeing时间格式
2週間前
humanleft
3de775eed1
test: 修复 MqttFaultConsumerTest 和 MqttStatusConsumerTest 编译与测试失败
- MqttFaultConsumerTest: 同步主代码重构后的方法签名
- insertAlarm 调用在主代码中已注释掉,移除对应 verify
- insertFault → insertAlertData (9 参数)
- updateFault → updateAlertData (7 参数)
- MqttStatusConsumerTest: 同步 MqttStatusConsumer 重构
- ValueOperations mock → HashOperations mock
- opsForValue().set() → opsForHash().putAll()
- 移除 fleet_id 相关测试数据(主代码已不再使用)
- CLAUDE.md: 更新 Redis key patterns 和 TDengine 连接说明
2週間前
humanleft
f36730933d
fix: substring 越界防护 + Redis key 解析修正
- VehicleSyncTask: 改用前缀精确切割解析 Redis key,避免 split 对含下划线 ID 的误解析
- VehicleSyncTask/MqttChargeStationConsumer/MqttDynamicConsumer: substring(0, 2) 增加长度校验
2週間前
humanleft
1874089606
test: 修复 mqy20260511 合并后的测试编译和运行时失败
2週間前
humanleft
806cdb0a64
deploy: taos-explorer 和 iot-platform 改用 host.containers.internal 访问 TDengine
- tdengine-operator 同时启动 taosd + taosadapter(host 网络)
- 删除单独的 taosadapter bridge 容器
- explorer REST API 地址改为 host.containers.internal:6041
- explorer 去掉 cluster_native(bridge 网络 native 连接不可用)
- iot-platform JDBC URL 改为 host.containers.internal:6031
- start-tdengine-operator.sh 不传 taosd 参数,禁用 keeper/explorer
2週間前
humanleft
4ccc79ca3e
fix
2週間前
humanleft
e1532193d2
refactor: TdEngineService 代码清理
2週間前
humanleft
a5b7e0823f
feat(deploy): 容器网络架构改造 + 配置纳入版本控制
网络架构:
- iot-platform: host -> bridge (新增 -p 8887:8887 端口映射)
- taos-explorer: bridge (新增 /mnt/taos-explorer-data 数据持久化)
- tdengine-operator: 保持 host 网络
配置更新:
- TDengine URL: 127.0.0.1 -> 172.21.185.173 (宿主机内网 IP)
- Redis host: localhost -> ${REDIS_HOST:localhost} 环境变量
- taos.cfg fqdn: 127.0.0.1 -> 172.21.185.173
- explorer.toml cluster: localhost -> 172.21.185.173
部署脚本:
- REMOTE_DIR: /opt/iot-platform -> /mnt/iot-platform
- 日志目录: /opt/iot-platform/logs -> /mnt/iot-platform/logs
- 去掉自动回滚逻辑,保留容器用于排查
- 新增 -p 8887:8887 端口映射
新增服务器配置文件到仓库:
- deploy/Dockerfile
- deploy/start-container.sh
- deploy/config/taos.cfg
- deploy/config/explorer.toml
- deploy/config/tdengine-explorer.conf
- .env (移除 .gitignore 忽略)
2週間前
humanleft
356fd0b43c
chore: 修改 TDengine JDBC URL 为 127.0.0.1:6031
- application.yml: localhost → 127.0.0.1 避免 IPv6 解析问题
- IotProperties.java: 同步修改默认 URL
2週間前
humanleft
bb26329b48
chore: 部署全新 TDengine 3.3.6.0 实例(端口 6031),升级 JDBC 驱动至 3.3.2
- 服务器新建独立 TDengine 容器实例(Podman),端口 6031
- 数据卷挂载至 /mnt/tdengine-new/(大容量数据盘)
- 部署 taos-explorer Web 控制台,nginx 6060 端口反向代理 + Basic Auth
- 升级 taos-jdbcdriver 3.2.7 → 3.3.2
- 应用 JDBC URL 指向新实例 localhost:6031
- TdEngineService 删除 device_id VARCHAR(32) 特殊处理,统一 VARCHAR(255)
2週間前
humanleft
216f58c8b6
超级表传输
2週間前
humanleft
04e70a5dea
格式
2週間前
humanleft
5c88b90a61
fix(service): 放宽表名校验正则,允许数字开头和连字符
2週間前
humanleft
72e23e493c
refactor(service): 统一 VARCHAR 长度为 255,移除 device_id 特殊处理
- DEFAULT_VARCHAR_LENGTH 38 → 255
- 删除 DEVICE_ID_VARCHAR_LENGTH 常量
- 删除 getColumnTypeForDDL 中 device_id 的 VARCHAR(32) 特殊分支
- 删除 formatValue 中 device_id 的 UUID 去连字符逻辑
- 同步更新相关单元测试断言
2週間前
humanleft
9313097021
chore: 删除 TdEngineService 中未使用的 closeQuietly 方法
2週間前
humanleft
1de1fcc14c
refactor(service): 全面优化 TdEngineService 资源管理与代码质量
- 资源管理:loadStableColumnsFromDB / ensureColumnsExist / ensureTableExists
统一改为 try-with-resources,消除手动 close 的冗长代码
- 缓存策略:getStableColumns 使用 computeIfAbsent 解决多线程竞态条件;
缓存满时改为淘汰最老的 20% 条目,替代粗暴 clear()
- 列数计算:insertBatchInternal 中扣除 existingColumns 与 columnTypes 的交集,
避免重复计算导致不必要的插入拒绝
- 缓存更新:ensureColumnsExist 循环结束后统一更新 stableColumnCache
- 连接池配置:IotProperties.TDengine 新增 HikariCP 配置项,initDataSource 改为读取配置
- 日志级别:loadStableColumnsFromDB 表不存在场景 error→debug;
formatValue 超长截断 info→debug
- ZoneOffset.of("+8") 提取为 static final ZONE_OFFSET_8,避免重复创建
2週間前
humanleft
2acb0f92d8
refactor(mqtt): 优化 MqttChargeStationConsumer 并补充单元测试
- 移除冗余的 deepCopyMap 调用(JSON 解析结果已是新对象)
- 空数据列表提前返回,避免无效的 topic 解析
- 简化空 map 过滤逻辑,消除误导性变量名 'list'
- 新增 4 个单元测试覆盖正常写入、空数据、无效 topic、过滤 null/空 map 场景
2週間前
humanleft
7a2edf5fa2
fix(mapper): 修复 SysControllerMapper XML id 与接口方法名不一致
selectjingweidu → selectCoordinates,与 Java 接口定义保持一致。
2週間前
humanleft
7a72c540e7
refactor(domain): Domain 类字段命名规范化(snake_case → camelCase)
- ControllerData: fleet_id→fleetId, controller_id→controllerId, cmd_topics→cmdTopics, fault_prot→faultProt,加 @JSONField 保持 JSON 序列化兼容,字段改为 private
- SysFault: 修复 getter/setter 内部引用旧字段名的问题
- MqttGenericConsumer/MqttFaultConsumer: 同步更新字段访问器调用
- TDengineServiceTest: 更新超长 VARCHAR 截断测试断言
2週間前
humanleft
9ef8ad942c
refactor: 全面规范化项目命名
变量名:
- weather → faultData(消除误导性命名)
- jingdu/weidu → longitude/latitude(消除中文拼音)
- jingduValue/weiduValue → longitudeValue/latitudeValue
- tdEngineshuju → tdEngineData(消除中文拼音)
- companyid → companyId
- controllername → controllerName
- tablename → tableNames
方法名(camelCase 规范化):
- selectall → selectAll
- insertfault → insertFault, updatefault → updateFault
- insertalarm → insertAlarm
- createmessage → createMessage
- selecttables → selectTables, inserttables → insertTables
- selectjingweidu → selectCoordinates(消除中文拼音)
- triggermethod → triggerMethod
- 等 40+ 个 mapper/service 方法名
类名:
- Sysrealtime → SysRealtime
- SysrealtimeMapper → SysRealtimeMapper
- SysrealtimeService → SysRealtimeService
- 所有引用同步更新
2週間前
humanleft
b8c75cd505
chore: 清理未使用的 Jackson ObjectMapper,调整测试可见性
- MqttFaultConsumerTest: 删除未使用的 ObjectMapper 实例和导入
- MqttFaultConsumer: insertTdEngine/triggermethod 改为 package-private
以支持同包测试访问
2週間前
humanleft
280890b8dc
fix(security): 修复 CRITICAL/HIGH 安全漏洞并全面优化代码质量
安全修复:
- MqttFaultConsumer: 修复 SSRF 漏洞,使用 UriComponentsBuilder + controllerId 白名单
- TdEngineService: 修复 SQL 注入,information_schema 查询改用 PreparedStatement
- TdEngineService: 收紧 ALLOWED_COLUMNS 正则,仅允许 ASCII 字母数字下划线
- TdEngineService: 修复静默数据丢失,超长字符串截断存储而非返回 NULL
- TdEngineAlarm: SQL 查询参数化,防止注入
- MqttFaultConsumer/MqttDynamicConsumer: topic/controllerId/dbName 白名单校验
- SysDeviceService: 添加表名校验
代码质量:
- 统一使用 Fastjson2 替代 Jackson ObjectMapper
- Domain 类字段全部改为 private(13 个类)
- topics.java 重命名为 Topics.java
- SimpleDateFormat 替换为线程安全的 DateTimeFormatter
- 提取魔法数字为常量(DEFAULT_BATCH_SIZE 等)
- SysWorkorderMapper.xml 修复 SQL 逻辑错误(AND/OR 括号)
- SysIndicatorsMapper.xml 修正字段名 createtime -> createdata
- SysControllerService 移除 Service 层 @Param 注解
- MqttGenericConsumer 提取过长方法
- NumericIdGenerator 支持 @Value 注入 workerId/dataCenterId
- 空 catch 块补充日志
日志级别优化:
- WARN 31 -> 9 (-71%),数据校验/格式问题降级为 INFO
- INFO 28 -> 50 (+79%),正常业务流使用 INFO
- ERROR 37 -> 33 (-11%),仅最终失败使用 ERROR
2週間前
humanleft
084000c8b8
del test
3週間前
humanleft
459ffcbb07
pom
3週間前
humanleft
eb72db0253
del
3週間前
humanleft
b8d3d8587c
chore: temporarily disable VehicleSyncTask to suppress MySQL table missing errors
3週間前
humanleft
ab5c6e7d5e
fix: explicitly name TDengineService bean as 'tdengineService'
4週間前
humanleft
a8d4903d34
fix: add @Qualifier to TDengineService constructor for explicit bean wiring
Without -parameters compiler flag, Spring cannot match ExecutorService
parameter by name when multiple beans of the same type exist. This
caused 'No bean named tdengineService available' startup failure on
production deployment.
4週間前
lenovo
e03f53003c
docs: update CLAUDE.md with security, MQTT, and lock improvements
4週間前
lenovo
3785254b58
docs: update CLAUDE.md with security, MQTT, and lock improvements
4週間前
humanleft
ebc7ce0564
refactor: remove dead code from VehicleSyncTask
- Remove unused service fields (sysWorkorderService, sysIndicatorsService, sysCompanyService)
- Remove commented-out insertIndicators() method
- Remove unused imports (SysCompany, LocalDate, DateTimeFormatter, @Autowired )
- Clean constructor to only keep actively used dependencies
4週間前
humanleft
470204fa50
docs: update CLAUDE.md with security, MQTT, and lock improvements
4週間前
humanleft
0fd6162ee1
refactor: CRITICAL/HIGH fixes — security, concurrency, resource leaks, input validation
Security:
- Remove hardcoded password fallbacks from application-druid.yml
- Add table name whitelist validation to SysFaultService & SysAlarmService
- Externalize VehicleSyncTask webhook URL to IotProperties config
Concurrency & Resource Management:
- Fix AbstractMqttConsumer reconnect/disconnect race (synchronized)
- Fix MqttClient resource leaks in disconnect() with separate try blocks
- Fix AbstractDynamicMqttConsumer broken-state MqttClient reuse
- Fix TDengineService stableColumnCache unbounded growth (MAX_CACHE_SIZE=1000)
Input Validation:
- Add null/empty checks to MqttGenericConsumer (controllerId, path, timestamp)
- Add null/empty checks to MqttFaultConsumer (controllerId, deviceId, type, desc)
- Guard against ArrayIndexOutOfBoundsException on topic path split
Tests:
- Update VehicleSyncTaskTest for new IotProperties constructor param
- All 79 tests pass, build succeeds
4週間前
humanleft
e6338b3ef4
refactor: P0-P1 complete — field injection, logging, hardcoded values, resource mgmt
P0 fixes:
- MqttFaultConsumer: array index bounds check, NPE guard on null coordinates
- MqttStatusConsumer: NPE guard on null required fields
- TDegnineAlarm: ResultSet try-with-resources, SQL injection whitelist
- AbstractMqttConsumer: brokerUrl parsing validation, mqttPassword null guard
P1 fixes:
- Field injection → constructor injection (7 files: MqttFaultConsumer,
MqttStatusConsumer, SysIndicatorsService, SysWorkorderService,
TDengineService, TDegnineAlarm, AbstractMqttConsumer)
- SLF4J {} placeholder logging (4 places in AbstractMqttConsumer)
- Extract hardcoded constants: alarm status, date format, company ID prefix,
webhook URL → IotProperties configuration
- TDengineService: dataSourceInitialized volatile, GZIP try-with-resources
Tests:
- Fix MqttStatusConsumerTest/MqttFaultConsumerTest/MqttGenericConsumerTest
for new constructors (remove @InjectMocks , manual instantiation)
- 65/67 tests passing (2 TDengine JNI env-dependent failures excluded)
4週間前
humanleft
28470869a3
test(P1): 新增3个测试类覆盖动态消费者基类与子类
- AbstractDynamicMqttConsumerTest: 4个测试覆盖 deepCopyMap(null/深拷贝/嵌套/List)
- MqttDynamicConsumerTest: 6个测试覆盖 fetchTopics + insertredis
- MqttChargeStationConsumerTest: 4个测试覆盖 fetchTopics(配置/null/空白/trim)
- 测试总数 65→79,JaCoCo覆盖率 18%→25%
4週間前
humanleft
5cbb11d169
chore(P2): 替换mysql驱动,移除emoji统一日志格式
- mysql-connector-java → mysql-connector-j 8.0.33
- TDengineService: 移除9处emoji(✅❌⚠️)
- 统一使用SLF4J {}占位符,无字符串拼接
4週間前
humanleft
7753e41899
refactor(P1): 字段注入→构造函数注入,扩展配置校验
- VehicleSyncTask: 10个@Autowired 字段改为final+构造函数注入
- MqttGenericConsumer: 3个@Autowired 字段改为构造函数参数
- IotProperties: validate()新增brokerUrl和url空值校验
- MqttGenericConsumerTest: 更新匹配4参数构造函数
4週間前
humanleft
66e22bd4a7
refactor(P0): 提取 AbstractDynamicMqttConsumer 基类消除400+行重复
- 新建 AbstractDynamicMqttConsumer 抽象基类,抽取连接/订阅/重连公共逻辑
- MqttDynamicConsumer extends 基类,仅保留 fetchTopics + processMessage
- MqttChargeStationConsumer extends 基类,仅保留 fetchTopics + processMessage
- 配置化 ChargeStation topic(IotProperties + application.yml)
- 修复 @DependsOn 位置(方法→类级别)
- 从 disconnect() 中移除 tdengineService.close()(Spring单例不应被消费者关闭)
- 基类日志统一使用 {} 占位符,移除 emoji
4週間前
humanleft
d0cff8b7f2
fix(P0): 5项安全与稳定性修复
- MqttGenericConsumer: destroy/init → refreshMqttSubscription() 避免破坏Spring生命周期
- MqttGenericConsumer: 添加 topics/cmdtopics/faultprot 空值保护防止NPE
- MqttGenericConsumer: 合并3处重复 persist 调用到方法末尾
- VehicleSyncTask: 修复 coordinateMap.get 的NPE风险(先判空再equals)
- VehicleSyncTask: 修复逗号分隔逻辑错误(i > size-1 → i < size-1)
- MqttDynamicConsumer: 删除废弃的 shutdownExecutor 死代码
4週間前
humanleft
c5351f22bf
fix: resolve merge conflict in MqttGenericConsumer
Remote branch overwrote the AbstractMqttConsumer inheritance structure.
Restored extends AbstractMqttConsumer + constructor injection while
preserving remote's business logic changes (updatecontrollerAccept,
SimpleDateFormat conversion, 6-arg insertsyscontroller).
4週間前
humanleft
8c8fe4baed
refactor: migrate MQTT Consumer thread pools to Spring-managed beans
- AbstractMqttConsumer: remove private executorService, use constructor injection
- MqttDynamicConsumer/MqttChargeStationConsumer: remove private core/write executors,
inject via @Qualifier from ExecutorConfig
- ExecutorConfig: add mqttCoreExecutor, mqttWriteExecutor, abstractConsumerExecutor
with destroyMethod="shutdown"
- VehicleSyncTask: refine scanKeys() exception handling, add per-record try-catch
in doUpdateSysCar() loop
- Remove unused jjwt 0.9.1 (CVE-2019-17195) and springfox-boot-starter 3.0.0 deps
4週間前
humanleft
a0d8471a2f
refactor: remove legacy RuoYi artifacts and unused dependencies
- Delete old RuoYi scripts: bin/*.bat, ry.bat, ry.sh
- Delete empty directories: controller/, service/impl/, static/, templates/
- Delete unused mybatis-config.xml (Spring Boot auto-configures MyBatis)
- Remove unused Maven dependencies: fastjson2, poi-ooxml
- Clean root pom.xml: remove bitwalker, kaptcha, oshi, velocity, poi, fastjson2 version management
Build verified: mvn clean package passes.
4週間前
humanleft
992ee7540f
refactor: strip RuoYi, create standalone iot-platform module
- Delete 6 legacy RuoYi modules (ruoyi-admin/framework/system/common/quartz/generator)
- Create new iot-platform Spring Boot 2.5.15 module with all IoT business
- Migrate 5 MQTT consumers, VehicleSyncTask, 14 services, 15 domains/mappers
- P0 security fixes:
- Replace string-concat SQL with field whitelist + escapeValue in TDengineService, DynamicSqlProvider, TDegnineAlarm
- Externalize hardcoded MQTT/TDengine credentials to application.yml via IotProperties
- Fix thread-unsafe static shared Connection/Statement in TDegnineAlarm (use per-method connections)
- Logging normalization: replace 148 System.out/err.println with SLF4J, add logback-spring.xml (100MB rolling, 30d retention)
- Dead code cleanup: NumericIdGenerator (238→65 lines), merge TDengineConnecting into TDengineService
- Verified: mvn clean package passes, app boots with all 5 MQTT consumers
1ヶ月前
humanleft
4e57215f6f
refactor: P0-P3 security hardening and quality overhaul
Security (P0):
- Remove hardcoded MQTT/TDengine credentials from IotProperties defaults
- Clear password fallbacks in application.yml (was ${ENV:real_password})
- Add @PostConstruct validation to block startup on missing credentials
- Add Redis distributed locks to VehicleSyncTask 4 @Scheduled methods
- Move static thread pools to Spring-managed beans with destroyMethod
Robustness (P1):
- Split broad catch(Exception) into specific types (MqttException,
InterruptedException, TimeoutException, ExecutionException)
- Restore InterruptedException flag in all catch blocks
- Add closeQuietly() for JDBC Statement/ResultSet cleanup
- Configure RestTemplate with 5s connect / 10s read timeouts
Quality (P2):
- Production log levels: debug/trace → info
- Tomcat: max 800→200, min-spare 100→20, accept-count 1000→100
- Redis pool: max-active 8→16, max-idle 8→16, min-idle 0→4, max-wait 5s
Monitoring (P3):
- Actuator: show-details always→never, exposure health,info→health only
Testing:
- Add JaCoCo plugin for coverage reporting
- Add IotProperties, ExecutorConfig, RestTemplateConfig unit tests
- 26 tests passing
4週間前
humanleft
c3c35ae204
refactor: P0-P2 security, quality, deployment overhaul
Security (P0):
- Remove hardcoded MySQL password from SysrealtimeService
- Add table name whitelist validation (regex + length limit)
- Validate ${tableName} in SysrealtimeMapper.xml
- Externalize all credentials to .env file
- Fix unbounded thread pools in MQTT consumers
- Fix mysqlWritePool leak in MqttFaultConsumer
Quality (P1):
- Extract AbstractMqttConsumer base class (-500+ duplicate lines)
- Refactor VehicleSyncTask: SCAN instead of KEYS, batch ops,
RestTemplate timeout, loop-invariant extraction
- Add unit tests (RedisKeys, SysrealtimeService, MqttConsumer)
Architecture (P2):
- Add Spring Boot Actuator (/actuator/health)
- Create RedisKeys constants class for key namespace
- Add .env.example and start.sh for local development
Deployment:
- Add systemd service (iot-platform.service)
- Add deploy.sh with build, upload, backup, health check, rollback
- Add setup-server.sh for server initialization
- Add health-check.sh (Actuator + TCP port fallback)
Docs:
- Update CLAUDE.md for standalone iot-platform architecture
- Update README.md with build/deploy instructions
- Add deploy/README.md
4週間前
humanleft
af91b4c759
refactor: strip RuoYi, create standalone iot-platform module
- Delete 6 legacy RuoYi modules (ruoyi-admin/framework/system/common/quartz/generator)
- Create new iot-platform Spring Boot 2.5.15 module with all IoT business
- Migrate 5 MQTT consumers, VehicleSyncTask, 14 services, 15 domains/mappers
- P0 security fixes:
- Replace string-concat SQL with field whitelist + escapeValue in TDengineService, DynamicSqlProvider, TDegnineAlarm
- Externalize hardcoded MQTT/TDengine credentials to application.yml via IotProperties
- Fix thread-unsafe static shared Connection/Statement in TDegnineAlarm (use per-method connections)
- Logging normalization: replace 148 System.out/err.println with SLF4J, add logback-spring.xml (100MB rolling, 30d retention)
- Dead code cleanup: NumericIdGenerator (238→65 lines), merge TDengineConnecting into TDengineService
- Verified: mvn clean package passes, app boots with all 5 MQTT consumers
1ヶ月前
humanleft
9a4dca84c5
refactor: strip RuoYi, create standalone iot-platform module
- Delete 6 legacy RuoYi modules (ruoyi-admin/framework/system/common/quartz/generator)
- Create new iot-platform Spring Boot 2.5.15 module with all IoT business
- Migrate 5 MQTT consumers, VehicleSyncTask, 14 services, 15 domains/mappers
- P0 security fixes:
- Replace string-concat SQL with field whitelist + escapeValue in TDengineService, DynamicSqlProvider, TDegnineAlarm
- Externalize hardcoded MQTT/TDengine credentials to application.yml via IotProperties
- Fix thread-unsafe static shared Connection/Statement in TDegnineAlarm (use per-method connections)
- Logging normalization: replace 148 System.out/err.println with SLF4J, add logback-spring.xml (100MB rolling, 30d retention)
- Dead code cleanup: NumericIdGenerator (238→65 lines), merge TDengineConnecting into TDengineService
- Verified: mvn clean package passes, app boots with all 5 MQTT consumers
1ヶ月前