Communication-efficient federated learning with stagewise training strategy.

The efficiency of communication across workers is a significant factor that affects the performance of federated learning. Though periodic communication strategy is applied to reduce communication rounds in training, the communication cost is still high when the training data distributions are not independently and identically distributed (non-IID) which is common in federated learning. Recently, some works introduce variance reduction to eliminate the effect caused by non-IID data among workers. Nevertheless the provable optimal communication complexity O(log(ST)) and convergence rate O(1/(ST)) cannot be achieved simultaneously, where S denotes the number of sampled workers in each round and T is the number of iterations. To deal with this dilemma, we propose an optimization algorithm SQUARFA that adopts stagewise training framework coupling with variance reduction and uses a quick-start phase in each loop. Theoretical results show that SQUARFA achieves both optimal convergence rate and communication complexity for both strongly convex objectives and non-convex objectives under PL condition, thus fills the gap mentioned above. Then, a variant of SQUARFA yields the optimal theoretical results for general non-convex objectives. We further extend the technique in SQUARFA to the large batch setting and achieve optimal communication complexity. Experimental results demonstrate the superiority of the proposed algorithms.

Development of polymorphic EST-SSR markers and characterization of the autotetraploid genome of sainfoin (Onobrychis viciifolia).

BACKGROUND: Sainfoin (Onobrychis viciifolia) is a highly nutritious, tannin-containing, and tetraploid forage legume. Due to the lack of detailed transcriptomic and genomic information on this species, genetic and breeding projects for sainfoin improvement have been significantly hindered. METHODS: In this study, a total of 24,630,711 clean reads were generated from 14 different sainfoin tissues using Illumina paired-end sequencing technology and deposited in the NCBI SRA database (SRX3763386). From these clean reads, 77,764 unigene sequences were obtained and 6,752 EST-SSRs were identified using de novo assembly. A total of 2,469 primer pairs were designed, and 200 primer pairs were randomly selected to analyze the polymorphism in five sainfoin wild accessions. RESULTS: Further analysis of 40 sainfoin individuals from the five wild populations using 61 EST-SSR loci showed that the number of alleles per locus ranged from 4 to 15, and the expected heterozygosity varied from 0.55 to 0.91. Additionally, by counting the EST-SSR band number and sequencing the three or four bands in one sainfoin individual, sainfoin was confirmed to be autotetraploid. This finding provides a high level of information about this plant. DISCUSSION: Through this study, 61 EST-SSR markers were successfully developed and shown to be useful for genetic studies and investigations of population genetic structures and variabilities among different sainfoin accessions.