任务参数“IJobHandler.execute”入参改为“String params”,增强入参通用性。
脚本任务Shell、Python和Nodejs等支持获取分片参数;
This commit is contained in:
parent
6042011dc7
commit
14d9d6a785
|
@ -791,11 +791,21 @@ XXL-JOB会为每次调度请求生成一个单独的日志文件,需要通过
|
||||||
|
|
||||||
"分片广播" 以执行器为维度进行分片,支持动态扩容执行器集群从而动态增加分片数量,协同进行业务处理;在进行大数据量业务操作时可显著提升任务处理能力和速度。
|
"分片广播" 以执行器为维度进行分片,支持动态扩容执行器集群从而动态增加分片数量,协同进行业务处理;在进行大数据量业务操作时可显著提升任务处理能力和速度。
|
||||||
|
|
||||||
"分片广播" 和普通任务开发流程一致,不同之处在于可以可以获取分片参数,获取分片参数对象的代码如下(可参考Sample示例执行器中的示例任务"ShardingJobHandler" ):
|
"分片广播" 和普通任务开发流程一致,不同之处在于可以可以获取分片参数,获取分片参数进行分片业务处理。
|
||||||
|
|
||||||
ShardingUtil.ShardingVO shardingVO = ShardingUtil.getShardingVo();
|
- Java语言任务获取分片参数方式:BEAN、GLUE模式(Java)
|
||||||
|
```
|
||||||
|
// 可参考Sample示例执行器中的示例任务"ShardingJobHandler"了解试用
|
||||||
|
ShardingUtil.ShardingVO shardingVO = ShardingUtil.getShardingVo();
|
||||||
|
```
|
||||||
|
- 脚本语言任务获取分片参数方式:GLUE模式(Shell)、GLUE模式(Python)、GLUE模式(Nodejs)
|
||||||
|
```
|
||||||
|
// 脚本任务入参固定为三个,依次为:任务传参、分片序号、分片总数。以Shell模式任务为例,获取分片参数代码如下
|
||||||
|
echo "分片序号 index = $2"
|
||||||
|
echo "分片总数 total = $3"
|
||||||
|
```
|
||||||
|
|
||||||
该分片参数对象拥有两个属性:
|
分片参数属性说明:
|
||||||
|
|
||||||
index:当前分片序号(从0开始),执行器集群列表中当前执行器的序号;
|
index:当前分片序号(从0开始),执行器集群列表中当前执行器的序号;
|
||||||
total:总分片数,执行器集群的总机器数量;
|
total:总分片数,执行器集群的总机器数量;
|
||||||
|
@ -1083,7 +1093,8 @@ Tips: 历史版本(V1.3.x)目前已经Release至稳定版本, 进入维护阶段
|
||||||
- 18、任务列表界面,左侧菜单合并时样式错乱问题修复;
|
- 18、任务列表界面,左侧菜单合并时样式错乱问题修复;
|
||||||
- 19、执行器地址录入交互优化,地址长度扩展支持至512位,支持大规模执行器集群配置;
|
- 19、执行器地址录入交互优化,地址长度扩展支持至512位,支持大规模执行器集群配置;
|
||||||
- 20、项目依赖升级,如spring、jackson等;
|
- 20、项目依赖升级,如spring、jackson等;
|
||||||
|
- 21、任务参数“IJobHandler.execute”入参改为“String params”,增强入参通用性。
|
||||||
|
- 22、脚本任务Shell、Python和Nodejs等支持获取分片参数;
|
||||||
|
|
||||||
### TODO LIST
|
### TODO LIST
|
||||||
- 1、任务权限管理:执行器为粒度分配权限,核心操作校验权限;
|
- 1、任务权限管理:执行器为粒度分配权限,核心操作校验权限;
|
||||||
|
@ -1099,10 +1110,10 @@ Tips: 历史版本(V1.3.x)目前已经Release至稳定版本, 进入维护阶段
|
||||||
- 11、任务日志,记录发起调度的机器信息;
|
- 11、任务日志,记录发起调度的机器信息;
|
||||||
- 12、任务告警逻辑调整:任务调度,以及任务回调失败时,均推送监控队列。后期考虑通过任务Log字段控制告警状态;
|
- 12、任务告警逻辑调整:任务调度,以及任务回调失败时,均推送监控队列。后期考虑通过任务Log字段控制告警状态;
|
||||||
- 13、执行器Log清理功能:调度中心Log删除时同步删除执行器中的Log文件;
|
- 13、执行器Log清理功能:调度中心Log删除时同步删除执行器中的Log文件;
|
||||||
- 14、脚本任务 Shell、Python和Nodejs,如何友好获取分片参数;
|
- 14、Bean模式任务,JobHandler自动从执行器中查询展示为下拉框,选择后自动填充任务名称等属性;
|
||||||
- 15、Bean模式任务,JobHandler自动从执行器中查询展示为下拉框,选择后自动填充任务名称等属性;
|
- 15、任务告警邮件优化,调整为表格形式;
|
||||||
- 16、任务告警邮件优化,调整为表格形式;
|
- 16、执行器回调地址格式兼容,支持非"/"结尾路径配置;
|
||||||
- 17、执行器回调地址格式兼容,支持非"/"结尾路径配置;
|
- 17、任务事件触发API服务优化,支持调用时动态传参;
|
||||||
|
|
||||||
|
|
||||||
## 七、其他
|
## 七、其他
|
||||||
|
|
|
@ -156,7 +156,7 @@ CREATE TABLE `XXL_JOB_QRTZ_TRIGGER_INFO` (
|
||||||
`alarm_email` varchar(255) DEFAULT NULL COMMENT '报警邮件',
|
`alarm_email` varchar(255) DEFAULT NULL COMMENT '报警邮件',
|
||||||
`executor_route_strategy` varchar(50) DEFAULT NULL COMMENT '执行器路由策略',
|
`executor_route_strategy` varchar(50) DEFAULT NULL COMMENT '执行器路由策略',
|
||||||
`executor_handler` varchar(255) DEFAULT NULL COMMENT '执行器任务handler',
|
`executor_handler` varchar(255) DEFAULT NULL COMMENT '执行器任务handler',
|
||||||
`executor_param` varchar(255) DEFAULT NULL COMMENT '执行器任务参数',
|
`executor_param` varchar(512) DEFAULT NULL COMMENT '执行器任务参数',
|
||||||
`executor_block_strategy` varchar(50) DEFAULT NULL COMMENT '阻塞处理策略',
|
`executor_block_strategy` varchar(50) DEFAULT NULL COMMENT '阻塞处理策略',
|
||||||
`executor_fail_strategy` varchar(50) DEFAULT NULL COMMENT '失败处理策略',
|
`executor_fail_strategy` varchar(50) DEFAULT NULL COMMENT '失败处理策略',
|
||||||
`glue_type` varchar(50) NOT NULL COMMENT 'GLUE类型',
|
`glue_type` varchar(50) NOT NULL COMMENT 'GLUE类型',
|
||||||
|
@ -174,7 +174,7 @@ CREATE TABLE `XXL_JOB_QRTZ_TRIGGER_LOG` (
|
||||||
`glue_type` varchar(50) DEFAULT NULL COMMENT 'GLUE类型',
|
`glue_type` varchar(50) DEFAULT NULL COMMENT 'GLUE类型',
|
||||||
`executor_address` varchar(255) DEFAULT NULL COMMENT '执行器地址,本次执行的地址',
|
`executor_address` varchar(255) DEFAULT NULL COMMENT '执行器地址,本次执行的地址',
|
||||||
`executor_handler` varchar(255) DEFAULT NULL COMMENT '执行器任务handler',
|
`executor_handler` varchar(255) DEFAULT NULL COMMENT '执行器任务handler',
|
||||||
`executor_param` varchar(255) DEFAULT NULL COMMENT 'executor_param',
|
`executor_param` varchar(512) DEFAULT NULL COMMENT '执行器任务参数',
|
||||||
`trigger_time` datetime DEFAULT NULL COMMENT '调度-时间',
|
`trigger_time` datetime DEFAULT NULL COMMENT '调度-时间',
|
||||||
`trigger_code` varchar(255) NOT NULL DEFAULT '0' COMMENT '调度-结果',
|
`trigger_code` varchar(255) NOT NULL DEFAULT '0' COMMENT '调度-结果',
|
||||||
`trigger_msg` varchar(2048) DEFAULT NULL COMMENT '调度-日志',
|
`trigger_msg` varchar(2048) DEFAULT NULL COMMENT '调度-日志',
|
||||||
|
|
|
@ -143,7 +143,7 @@
|
||||||
</div>
|
</div>
|
||||||
<div class="form-group">
|
<div class="form-group">
|
||||||
<label for="firstname" class="col-sm-2 control-label">执行参数<font color="black">*</font></label>
|
<label for="firstname" class="col-sm-2 control-label">执行参数<font color="black">*</font></label>
|
||||||
<div class="col-sm-4"><input type="text" class="form-control" name="executorParam" placeholder="请输入“执行参数”" maxlength="100" ></div>
|
<div class="col-sm-4"><input type="text" class="form-control" name="executorParam" placeholder="请输入“执行参数”" maxlength="512" ></div>
|
||||||
<label for="lastname" class="col-sm-2 control-label">子任务Key<font color="black">*</font></label>
|
<label for="lastname" class="col-sm-2 control-label">子任务Key<font color="black">*</font></label>
|
||||||
<div class="col-sm-4"><input type="text" class="form-control" name="childJobKey" placeholder="请输入子任务的任务Key,如存在多个逗号分隔" maxlength="100" ></div>
|
<div class="col-sm-4"><input type="text" class="form-control" name="childJobKey" placeholder="请输入子任务的任务Key,如存在多个逗号分隔" maxlength="100" ></div>
|
||||||
</div>
|
</div>
|
||||||
|
@ -192,7 +192,7 @@ import com.xxl.job.core.handler.IJobHandler;
|
||||||
public class DemoGlueJobHandler extends IJobHandler {
|
public class DemoGlueJobHandler extends IJobHandler {
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public ReturnT<String> execute(String... params) throws Exception {
|
public ReturnT<String> execute(String param) throws Exception {
|
||||||
XxlJobLogger.log("XXL-JOB, Hello World.");
|
XxlJobLogger.log("XXL-JOB, Hello World.");
|
||||||
return ReturnT.SUCCESS;
|
return ReturnT.SUCCESS;
|
||||||
}
|
}
|
||||||
|
@ -300,7 +300,7 @@ process.exit(0)
|
||||||
</div>
|
</div>
|
||||||
<div class="form-group">
|
<div class="form-group">
|
||||||
<label for="firstname" class="col-sm-2 control-label">执行参数<font color="black">*</font></label>
|
<label for="firstname" class="col-sm-2 control-label">执行参数<font color="black">*</font></label>
|
||||||
<div class="col-sm-4"><input type="text" class="form-control" name="executorParam" placeholder="请输入“执行参数”" maxlength="100" ></div>
|
<div class="col-sm-4"><input type="text" class="form-control" name="executorParam" placeholder="请输入“执行参数”" maxlength="512" ></div>
|
||||||
<label for="lastname" class="col-sm-2 control-label">子任务Key<font color="black">*</font></label>
|
<label for="lastname" class="col-sm-2 control-label">子任务Key<font color="black">*</font></label>
|
||||||
<div class="col-sm-4"><input type="text" class="form-control" name="childJobKey" placeholder="请输入子任务的任务Key,如存在多个逗号分隔" maxlength="100" ></div>
|
<div class="col-sm-4"><input type="text" class="form-control" name="childJobKey" placeholder="请输入子任务的任务Key,如存在多个逗号分隔" maxlength="100" ></div>
|
||||||
</div>
|
</div>
|
||||||
|
|
|
@ -21,11 +21,11 @@ public abstract class IJobHandler {
|
||||||
/**
|
/**
|
||||||
* execute handler
|
* execute handler
|
||||||
*
|
*
|
||||||
* @param params
|
* @param param
|
||||||
* @return
|
* @return
|
||||||
* @throws Exception
|
* @throws Exception
|
||||||
*/
|
*/
|
||||||
public abstract ReturnT<String> execute(String... params) throws Exception;
|
public abstract ReturnT<String> execute(String param) throws Exception;
|
||||||
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|
|
@ -21,9 +21,9 @@ public class GlueJobHandler extends IJobHandler {
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public ReturnT<String> execute(String... params) throws Exception {
|
public ReturnT<String> execute(String param) throws Exception {
|
||||||
XxlJobLogger.log("----------- glue.version:"+ glueUpdatetime +" -----------");
|
XxlJobLogger.log("----------- glue.version:"+ glueUpdatetime +" -----------");
|
||||||
return jobHandler.execute(params);
|
return jobHandler.execute(param);
|
||||||
}
|
}
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|
|
@ -6,6 +6,7 @@ import com.xxl.job.core.handler.IJobHandler;
|
||||||
import com.xxl.job.core.log.XxlJobFileAppender;
|
import com.xxl.job.core.log.XxlJobFileAppender;
|
||||||
import com.xxl.job.core.log.XxlJobLogger;
|
import com.xxl.job.core.log.XxlJobLogger;
|
||||||
import com.xxl.job.core.util.ScriptUtil;
|
import com.xxl.job.core.util.ScriptUtil;
|
||||||
|
import com.xxl.job.core.util.ShardingUtil;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Created by xuxueli on 17/4/27.
|
* Created by xuxueli on 17/4/27.
|
||||||
|
@ -29,7 +30,7 @@ public class ScriptJobHandler extends IJobHandler {
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public ReturnT<String> execute(String... params) throws Exception {
|
public ReturnT<String> execute(String param) throws Exception {
|
||||||
|
|
||||||
if (!glueType.isScript()) {
|
if (!glueType.isScript()) {
|
||||||
return new ReturnT<String>(IJobHandler.FAIL.getCode(), "glueType["+ glueType +"] invalid.");
|
return new ReturnT<String>(IJobHandler.FAIL.getCode(), "glueType["+ glueType +"] invalid.");
|
||||||
|
@ -50,9 +51,16 @@ public class ScriptJobHandler extends IJobHandler {
|
||||||
// log file
|
// log file
|
||||||
String logFileName = XxlJobFileAppender.contextHolder.get();
|
String logFileName = XxlJobFileAppender.contextHolder.get();
|
||||||
|
|
||||||
|
// script params:0=param、1=分片序号、2=分片总数
|
||||||
|
ShardingUtil.ShardingVO shardingVO = ShardingUtil.getShardingVo();
|
||||||
|
String[] scriptParams = new String[3];
|
||||||
|
scriptParams[0] = param;
|
||||||
|
scriptParams[1] = String.valueOf(shardingVO.getIndex());
|
||||||
|
scriptParams[2] = String.valueOf(shardingVO.getTotal());
|
||||||
|
|
||||||
// invoke
|
// invoke
|
||||||
XxlJobLogger.log("----------- script file:"+ scriptFileName +" -----------");
|
XxlJobLogger.log("----------- script file:"+ scriptFileName +" -----------");
|
||||||
int exitValue = ScriptUtil.execToFile(cmd, scriptFileName, logFileName, params);
|
int exitValue = ScriptUtil.execToFile(cmd, scriptFileName, logFileName, scriptParams);
|
||||||
ReturnT<String> result = (exitValue==0)?IJobHandler.SUCCESS:new ReturnT<String>(IJobHandler.FAIL.getCode(), "script exit value("+exitValue+") is failed");
|
ReturnT<String> result = (exitValue==0)?IJobHandler.SUCCESS:new ReturnT<String>(IJobHandler.FAIL.getCode(), "script exit value("+exitValue+") is failed");
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
|
@ -114,19 +114,14 @@ public class JobThread extends Thread{
|
||||||
idleTimes = 0;
|
idleTimes = 0;
|
||||||
triggerLogIdSet.remove(triggerParam.getLogId());
|
triggerLogIdSet.remove(triggerParam.getLogId());
|
||||||
|
|
||||||
// parse param
|
|
||||||
String[] handlerParams = (triggerParam.getExecutorParams()!=null && triggerParam.getExecutorParams().trim().length()>0)
|
|
||||||
? (String[])(Arrays.asList(triggerParam.getExecutorParams().split(",")).toArray()) : null;
|
|
||||||
|
|
||||||
|
|
||||||
// log filename, like "logPath/yyyy-MM-dd/9999.log"
|
// log filename, like "logPath/yyyy-MM-dd/9999.log"
|
||||||
String logFileName = XxlJobFileAppender.makeLogFileName(new Date(triggerParam.getLogDateTim()), triggerParam.getLogId());
|
String logFileName = XxlJobFileAppender.makeLogFileName(new Date(triggerParam.getLogDateTim()), triggerParam.getLogId());
|
||||||
XxlJobFileAppender.contextHolder.set(logFileName);
|
XxlJobFileAppender.contextHolder.set(logFileName);
|
||||||
ShardingUtil.setShardingVo(new ShardingUtil.ShardingVO(triggerParam.getBroadcastIndex(), triggerParam.getBroadcastTotal()));
|
ShardingUtil.setShardingVo(new ShardingUtil.ShardingVO(triggerParam.getBroadcastIndex(), triggerParam.getBroadcastTotal()));
|
||||||
|
|
||||||
// execute
|
// execute
|
||||||
XxlJobLogger.log("<br>----------- xxl-job job execute start -----------<br>----------- Params:" + Arrays.toString(handlerParams));
|
XxlJobLogger.log("<br>----------- xxl-job job execute start -----------<br>----------- Param:" + triggerParam.getExecutorParams());
|
||||||
executeResult = handler.execute(handlerParams);
|
executeResult = handler.execute(triggerParam.getExecutorParams());
|
||||||
if (executeResult == null) {
|
if (executeResult == null) {
|
||||||
executeResult = IJobHandler.FAIL;
|
executeResult = IJobHandler.FAIL;
|
||||||
}
|
}
|
||||||
|
|
|
@ -20,7 +20,7 @@ import java.util.concurrent.TimeUnit;
|
||||||
public class DemoJobHandler extends IJobHandler {
|
public class DemoJobHandler extends IJobHandler {
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public ReturnT<String> execute(String... params) throws Exception {
|
public ReturnT<String> execute(String param) throws Exception {
|
||||||
XxlJobLogger.log("XXL-JOB, Hello World.");
|
XxlJobLogger.log("XXL-JOB, Hello World.");
|
||||||
|
|
||||||
for (int i = 0; i < 5; i++) {
|
for (int i = 0; i < 5; i++) {
|
||||||
|
|
|
@ -14,7 +14,7 @@ import com.xxl.job.core.util.ShardingUtil;
|
||||||
public class ShardingJobHandler extends IJobHandler {
|
public class ShardingJobHandler extends IJobHandler {
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public ReturnT<String> execute(String... params) throws Exception {
|
public ReturnT<String> execute(String param) throws Exception {
|
||||||
|
|
||||||
// 分片参数
|
// 分片参数
|
||||||
ShardingUtil.ShardingVO shardingVO = ShardingUtil.getShardingVo();
|
ShardingUtil.ShardingVO shardingVO = ShardingUtil.getShardingVo();
|
||||||
|
|
|
@ -25,7 +25,7 @@ import java.util.concurrent.TimeUnit;
|
||||||
public class DemoJobHandler extends IJobHandler {
|
public class DemoJobHandler extends IJobHandler {
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public ReturnT<String> execute(String... params) throws Exception {
|
public ReturnT<String> execute(String param) throws Exception {
|
||||||
XxlJobLogger.log("XXL-JOB, Hello World.");
|
XxlJobLogger.log("XXL-JOB, Hello World.");
|
||||||
|
|
||||||
for (int i = 0; i < 5; i++) {
|
for (int i = 0; i < 5; i++) {
|
||||||
|
|
|
@ -18,7 +18,7 @@ import org.springframework.stereotype.Service;
|
||||||
public class ShardingJobHandler extends IJobHandler {
|
public class ShardingJobHandler extends IJobHandler {
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public ReturnT<String> execute(String... params) throws Exception {
|
public ReturnT<String> execute(String param) throws Exception {
|
||||||
|
|
||||||
// 分片参数
|
// 分片参数
|
||||||
ShardingUtil.ShardingVO shardingVO = ShardingUtil.getShardingVo();
|
ShardingUtil.ShardingVO shardingVO = ShardingUtil.getShardingVo();
|
||||||
|
|
|
@ -25,7 +25,7 @@ import java.util.concurrent.TimeUnit;
|
||||||
public class DemoJobHandler extends IJobHandler {
|
public class DemoJobHandler extends IJobHandler {
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public ReturnT<String> execute(String... params) throws Exception {
|
public ReturnT<String> execute(String param) throws Exception {
|
||||||
XxlJobLogger.log("XXL-JOB, Hello World.");
|
XxlJobLogger.log("XXL-JOB, Hello World.");
|
||||||
|
|
||||||
for (int i = 0; i < 5; i++) {
|
for (int i = 0; i < 5; i++) {
|
||||||
|
|
|
@ -18,7 +18,7 @@ import org.springframework.stereotype.Service;
|
||||||
public class ShardingJobHandler extends IJobHandler {
|
public class ShardingJobHandler extends IJobHandler {
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public ReturnT<String> execute(String... params) throws Exception {
|
public ReturnT<String> execute(String param) throws Exception {
|
||||||
|
|
||||||
// 分片参数
|
// 分片参数
|
||||||
ShardingUtil.ShardingVO shardingVO = ShardingUtil.getShardingVo();
|
ShardingUtil.ShardingVO shardingVO = ShardingUtil.getShardingVo();
|
||||||
|
|
Loading…
Reference in New Issue