Off-policy two-dimensional reinforcement learning for optimal tracking control of batch processes with network-induced dropout and disturbances.

In this paper, a new off-policy two-dimensional (2D) reinforcement learning approach is proposed to deal with the optimal tracking control (OTC) issue of batch processes with network-induced dropout and disturbances. A dropout 2D augmented Smith predictor is first devised to estimate the present extended state utilizing past data of time and batch orientations. The dropout 2D value function and Q-function are further defined, and their relation is analyzed to meet the optimal performance. On this basis, the dropout 2D Bellman equation is derived according to the principle of the Q-function. For the sake of addressing the dropout 2D OTC problem of batch processes, two algorithms, i.e., the off-line 2D policy iteration algorithm and the off-policy 2D Q-learning algorithm, are presented. The latter method is developed by applying only the input and the estimated state, not the underlying information of the system. Meanwhile, the analysis with regard to the unbiasedness of solutions and convergence is separately given. The effectiveness of the provided methodologies is eventually validated through the application of a simulated case during the filling process.

Control of the injection velocity of embolic agents in embolization treatment.

BACKGROUND: Embolization is a common treatment method for tumor-targeting, anti-organ hyper-function, and hemostasis. However, the injection of embolic agents largely depends on the experiences of doctors, and doctors need to work in an X-ray environment that hurts their health. Even for a well-trained doctor, complications such as ectopic embolism caused by excessive embolic agents are always inevitable. RESULTS: This paper established a flow control curve model for embolic injection based on local arterial pressure. The end-vessel network was simplified as a porous media. The hemodynamic changes at different injection velocities and embolization degrees were simulated and analyzed. Sponge, a typical porous medium, was used to simulate the blocking and accumulation of embolic agents by capillary networks in the in vitro experimental platform. CONCLUSIONS: The simulation and experimental results show that the local arterial pressure is closely related to the critical injection velocity of the embolic agent reflux at a certain degree of embolization. The feasibility of this method for an automatic embolic injection system is discussed. It is concluded that the model of the flow control curve of embolic injection can effectively reduce the risk of ectopic embolism and shorten the time of embolic injection. The clinical application of this model is of great value in reducing radiation exposure and improving the success rate of interventional embolization.

Study Rheological Behavior of Polymer Solution in Different-Medium-Injection-Tools.

Previous studies showed the difficulty during polymer flooding and the low producing degree for the low permeability layer. To solve the problem, Daqing, the first oil company, puts forward the polymer-separate-layer-injection-technology which separates mass and pressure in a single pipe. This technology mainly increases the control range of injection pressure of fluid by using the annular de-pressure tool, and reasonably distributes the molecular weight of the polymer injected into the thin and poor layers through the shearing of the different-medium-injection-tools. This occurs, in order to take advantage of the shearing thinning property of polymer solution and avoid the energy loss caused by the turbulent flow of polymer solution due to excessive injection rate in different injection tools. Combining rheological property of polymer and local perturbation theory, a rheological model of polymer solution in different-medium-injection-tools is derived and the maximum injection velocity is determined. The ranges of polymer viscosity in different injection tools are mainly determined by the structures of the different injection tools. However, the value of polymer viscosity is mainly determined by the concentration of polymer solution. So, the relation between the molecular weight of polymer and the permeability of layers should be firstly determined, and then the structural parameter combination of the different-medium-injection-tool should be optimized. The results of the study are important for regulating polymer injection parameters in the oilfield which enhances the oil recovery with reduced the cost.

Effect of injection velocity and particle concentration on transport of nanoscale zero-valent iron and hydraulic conductivity in saturated porous media.

Successful groundwater remediation by injecting nanoscale zero-valent iron (NZVI) particles requires efficient particle transportation and distribution in the subsurface. This study focused on the influence of injection velocity and particle concentration on the spatial NZVI particle distribution, the deposition processes and on quantifying the induced decrease in hydraulic conductivity (K) as a result of particle retention by lab tests and numerical simulations. Horizontal column tests of 2m length were performed with initial Darcy injection velocities (q0) of 0.5, 1.5, and 4.1m/h and elemental iron input concentrations (Fe(0)in) of 0.6, 10, and 17g/L. Concentrations of Fe(0) in the sand were determined by magnetic susceptibility scans, which provide detailed Fe(0) distribution profiles along the column. NZVI particles were transported farther at higher injection velocity and higher input concentrations. K decreased by one order of magnitude during injection in all experiments, with a stronger decrease after reaching Fe(0) concentrations of about 14-18g/kg(sand). To simulate the observed nanoparticle transport behavior the existing finite-element code OGS has been successfully extended and parameterized for the investigated experiments using blocking, ripening, and straining as governing deposition processes. Considering parameter relationships deduced from single simulations for each experiment (e.g. deposition rate constants as a function of flow velocity) one mean parameter set has been generated reproducing the observations in an adequate way for most cases of the investigated realistic injection conditions. An assessment of the deposition processes related to clogging effects showed that the percentage of retention due to straining and ripening increased during experimental run time resulting in an ongoing reduction of K. Clogging is mainly evoked by straining which dominates particle deposition at higher flow velocities, while blocking and ripening play a significant role for attachment, mainly at lower injection velocities. Since the injection of fluids at real sites leads to descending flow velocities with increasing radial distance from the injection point, the simulation of particle transport requires accounting for all deposition processes mentioned above. Thus, the derived mean parameter set can be used as a basis for quantitative and predictive simulations of particle distributions and clogging effects at both lab and field scale. Since decreases in K can change the flow system, which may have positive as well as negative implications for the in situ remediation technology at a contaminated site, a reliable simulation is thus of great importance for NZVI injection and prediction.

Transport of carboxymethyl cellulose-coated zerovalent iron nanoparticles in a sand tank: Effects of sand grain size, nanoparticle concentration and injection velocity.

The transport of nanoscale zerovalent iron (NZVI) particles colloidally stabilized with 70,000 Da carboxymethyl cellulose (CMC), through sands with mean grain diameters of 180, 340 and 1140 µm (referred to as fine, intermediate and coarse-sized sand, respectively) was investigated in a 70-cm long, two-dimensional tank. The effect of NZVI concentrations (1 and 3 g-Fe L(-1)) and CMC concentrations (1 and 2 g L(-1)) and injection velocities (0.96 and 0.40 cm min(-1)) on particle transport were also evaluated with the intermediate sand. The overall NZVI mass fractions eluted from the tank were 36%, 25% and 16% in the coarse, intermediate and fine sands, respectively, when injected with 1 g L(-1) NZVI stabilized in 1 g L(-1) CMC. However, the mass fraction eluted reduced to 2.33% when the injection velocity was reduced from 0.96 to 0.40 cm min(-1) in the intermediate-sized sand. Maximum transport efficiency (38% NZVI mass eluted) in the intermediate-sized sand was achieved with 3 g L(-1) NZVI suspended in 2 g L(-1) CMC at an injection velocity of 0.96 cm min(-1). The transport efficiency was substantially decreased (11% NZVI mass eluted) when 3 g L(-1) NZVI was stabilized with only 1 g L(-1) CMC. The NZVI mean particle diameters in the porewaters remained unchanged at different locations in the tank suggesting that straining or gravity settling did not influence NZVI deposition. After NZVI injection, the hydraulic conductivity in the tank reduced by 80%-96%, depending on the CMC concentration and injection velocity.

The scanning effect of the injection flow rate of contrast medium on magnetic resonance image dynamic contrast-enhanced of prostate cancer / 实用医学杂志

Objective To investigate the effect of perfusion index of the injection flow rate of contrast medium on magnetic resonance image dynamic contrast-enhanced (DCE-MRI) of prostate cancer with different pathological grades. Methods Seventy patients with PCa、cardiac, normal renal function is and BMI≤25 kg/m2 were enrolled. The 2.5 mL/s, 5.0 mL/s dynamic enhanced injection velocity contrast agent was used for 35 patients and the reast 35 patients, respectively. All data was transferred to GE Advanced Workstation 4.3, and the indexes of the peripheral prostate cancerous zone were calculated by Functool2 of signal intensity time (SI-T), The time to minimum (Tmax), the whole enhancement degree (SImax%) and the maximum slope (Rmax) were calculated. The effect of different injection velocity on the dynamic enhanced perfusion index was analyzed. Results Tmax of pa-tients received 2.5 mL/s, 5.0 mL/s contrast agent injection velocity in the low risk group (Gleason score 2 to 6)、medium risk group (7 Gleason score) and high risk group (Gleason score 8 to 10) were (19.89 ± 2.76) s and (15.42 ± 1.68) s, (16.91 ± 2.34) s and (12.88 ± 1.73) s, (14.13 ± 1.81) s and (10.2 ± 1.42) s, with signifi-cant differences (t = 4.61, 3.1, 3.25, P < 0.01). The average SImax% of PCa in the two groups were (1.45 ± 0.17)%and (1.51 ± 0.27)%, (1.62 ± 0.12)%and (1.84 ± 0.18)%, (1.86 ± 0.16)% and (2.11 ± 0.28)%, Two groups of SImax% were statistically significant difference (t = -2.44, -4.55, -5.16, P < 0.05), respectively. The average Rmax of PCa of the two groups were (6.29 ± 2.62)% and (7.64 ± 4.09)%,(8.92 ± 4.21)% and (10.24 ± 9.09)%, (10.85 ± 2.89)% and (12.43 ± 3.51)%, with significant difference (t = -4.07,-3.85, -8.68, P <0.01). Tmax was shorter, SImax% and Rmax were higher of prostate cancer patients received 5.0 mL/s contrast agent injection velocity than those received 2.5 mL/s contrast agent injection velocity. Conclusion The dynamic enhancement perfusion index of prostate cancer patients received 5.0 mL/s contrast agent injection velocity is more sensitive than that of patients received 2.5 mL/s contrast agent injection velocity , which can improve the diagnosis of prostate cancer.