Residual Physics and Post-Posed Shielding for Safe Deep Reinforcement Learning Method.

Deep reinforcement learning (DRL) has been researched for computer room air conditioning unit control problems in data centers (DCs). However, two main issues limit the deployment of DRL in actual systems. First, a large amount of data is needed. Next, as a mission-critical system, safe control needs to be guaranteed, and temperatures in DCs should be kept within a certain operating range. To mitigate these issues, this article proposes a novel control method RP-SDRL. First, Residual Physics, built using the first law of thermodynamics, is integrated with the DRL algorithm and a Prediction Model. Subsequently, a Correction Model adapted from gradient descent is combined with the Prediction Model as Post-Posed Shielding to enforce safe actions. The RP-SDRL method was validated using simulation. Noise is added to the states of the model to further test its performance under state uncertainty. Experimental results show that the combination of Residual Physics and DRL can significantly improve the initial policy, sample efficiency, and robustness. Residual Physics can also improve the sample efficiency and the accuracy of the prediction model. While DRL alone cannot avoid constraint violations, RP-SDRL can detect unsafe actions and significantly reduce violations. Compared to the baseline controller, about 13% of electricity usage can be saved.

Addressing the quantitative conversion bottleneck in single-atom catalysis.

Single-atom catalysts (SACs) offer many advantages, such as atom economy and high chemoselectivity; however, their practical application in liquid-phase heterogeneous catalysis is hampered by the productivity bottleneck as well as catalyst leaching. Flow chemistry is a well-established method to increase the conversion rate of catalytic processes, however, SAC-catalysed flow chemistry in packed-bed type flow reactor is disadvantaged by low turnover number and poor stability. In this study, we demonstrate the use of fuel cell-type flow stacks enabled exceptionally high quantitative conversion in single atom-catalyzed reactions, as exemplified by the use of Pt SAC-on-MoS2/graphite felt catalysts incorporated in flow cell. A turnover frequency of approximately 8000 h-1 that corresponds to an aniline productivity of 5.8 g h-1 is achieved with a bench-top flow module (nominal reservoir volume of 1 cm3), with a Pt1-MoS2 catalyst loading of 1.5 g (3.2 mg of Pt). X-ray absorption fine structure spectroscopy combined with density functional theory calculations provide insights into stability and reactivity of single atom Pt supported in a pyramidal fashion on MoS2. Our study highlights the quantitative conversion bottleneck in SAC-mediated fine chemicals production can be overcome using flow chemistry.

Rapid Prototyping of Polymer-Based Rolled-Up Microfluidic Devices.

This work presents the simple and rapid fabrication of a polymer-based microfluidic prototype manufactured by rolling up thin films of polymer. The thin films were fabricated via a casting method and rolled up around a center core with the aid of plasma activation to create a three-dimensional (3D) spiral microchannel, hence reducing the time and cost of manufacture. In this work, rolled-up devices with single or dual fluidic networks fabricated from a single or two films were demonstrated for heat sink or heat exchanger applications, respectively. The experimental results show good heat transfer in the rolled-up system at various flow rates for both heat sink and heat exchanger devices, without any leakages. The rolled-up microfluidic system creates multiple curved channels, allowing for the generation of Dean vortices, which in turn lead to an enhancement of heat and mass transfer and prevention of fouling formation. These benefits enable the devices to be employed for many diverse applications, such as heat-transfer devices, micromixers, and sorters. To our knowledge, this work would be the first report on a microfluidic prototype of 3D spiral microchannel made from rolled-up polymeric thin film. This novel fabrication approach may represent the first step towards the development of a pioneering prototype for roll-to-roll processing, permitting the mass production of polymer-based microchannels from single or multiple thin films.

A pump-free microfluidic 3D perfusion platform for the efficient differentiation of human hepatocyte-like cells.

The practical application of microfluidic liver models for in vitro drug testing is partly hampered by their reliance on human primary hepatocytes, which are limited in number and have batch-to-batch variation. Human stem cell-derived hepatocytes offer an attractive alternative cell source, although their 3D differentiation and maturation in a microfluidic platform have not yet been demonstrated. We develop a pump-free microfluidic 3D perfusion platform to achieve long-term and efficient differentiation of human liver progenitor cells into hepatocyte-like cells (HLCs). The device contains a micropillar array to immobilize cells three-dimensionally in a central cell culture compartment flanked by two side perfusion channels. Constant pump-free medium perfusion is accomplished by controlling the differential heights of horizontally orientated inlet and outlet media reservoirs. Computational fluid dynamic simulation is used to estimate the hydrostatic pressure heads required to achieve different perfusion flow rates, which are experimentally validated by micro-particle image velocimetry, as well as viability and functional assessments in a primary rat hepatocyte model. We perform on-chip differentiation of HepaRG, a human bipotent progenitor cell, and discover that 3D microperfusion greatly enhances the hepatocyte differentiation efficiency over static 2D and 3D cultures. However, HepaRG progenitor cells are highly sensitive to the time-point at which microperfusion is applied. Isolated HepaRG cells that are primed as static 3D spheroids before being subjected to microperfusion yield a significantly higher proportion of HLCs (92%) than direct microperfusion of isolated HepaRG cells (62%). This platform potentially offers a simple and efficient means to develop highly functional microfluidic liver models incorporating human stem cell-derived HLCs. Biotechnol. Bioeng. 2017;114: 2360-2370. © 2017 Wiley Periodicals, Inc.