spring batch remote partition with multiple tables -
i have developed 1 app have process more 5 million records 1 large table. have used spring batch remote partitioning working fine.
i have use same job , tweak create partition multiple table. each partition have table name , min , max records slave knows table has read data , prepare file.
my question is, can done if yes there sample out there on github.
thank you
yes can done. you'll need implement own partitioner handle logic of creating executioncontexts each table/row range.
a couple resources:
partitionerinterface documentation: http://docs.spring.io/spring-batch/apidocs/org/springframework/batch/core/partition/support/partitioner.html- section 7.4 of spring batch's scaling , parallel processing chapter: http://docs.spring.io/spring-batch/trunk/reference/html/scalability.html
- my introduction spring batch video covers partitioning: https://www.youtube.com/watch?v=cytj5yt7czu , related github repository: https://github.com/mminella/spring-batch-talk-2.0
Comments
Post a Comment