Learning Multi-Agent Cooperation via Considering Actions of Teammates.

Recently value-based centralized training with decentralized execution (CTDE) multi-agent reinforcement learning (MARL) methods have achieved excellent performance in cooperative tasks. However, the most representative method among these methods, Q-network MIXing (QMIX), restricts the joint action Q values to be a monotonic mixing of each agent's utilities. Furthermore, current methods cannot generalize to unseen environments or different agent configurations, which is known as ad hoc team play situation. In this work, we propose a novel Q values decomposition that considers both the return of an agent acting on its own and cooperating with other observable agents to address the nonmonotonic problem. Based on the decomposition, we propose a greedy action searching method that can improve exploration and is not affected by changes in observable agents or changes in the order of agents' actions. In this way, our method can adapt to ad hoc team play situation. Furthermore, we utilize an auxiliary loss related to environmental cognition consistency and a modified prioritized experience replay (PER) buffer to assist training. Our extensive experimental results show that our method achieves significant performance improvements in both challenging monotonic and nonmonotonic domains, and can handle the ad hoc team play situation perfectly.

First-principles determination of the crystallography of the low-temperature phase for NbRu and TaRu shape memory alloys.

The crystallography of the low-temperature phases (ß'') for shape memory alloys (NbRu and TaRu) has been debated for decades. Though a P2/m monoclinic structure has been proposed for the ß'' phase, the proposed structure is not able to completely represent the measured diffraction data. In this work, the crystallography of the ß'' phase was investigated by first-principles calculations. We showed that the previously reported P2/m monoclinic structure was lattice unstable due to the presence of the soft phonon mode. A P21/m monoclinic structure was derived from the P2/m monoclinic structure by displacing its atoms according to the eigenvector of the soft phonon mode at the Γ point. The P21/m and the P2/m monoclinic structures are structurally similar, but the former one is energetically and structurally more favorable than the latter one. We concluded that the ß'' phase preferred to crystallize in the P21/m monoclinic structure rather than the previously reported P2/m monoclinic structure. Our results offer guidance for the experimental determination of the crystallography of the ß'' phase for NbRu and TaRu.

Strain effects in the electron orbital coupling and electric structure of graphene.

Graphene is not only a very strong two-dimensional material, but is also able to sustain reversible tensile elastic strain larger than 20%, which yields an interesting possibility to regulate the properties of graphene by applied strain. We have investigated the strain effects in the electron orbital coupling and electric structure of graphene adopting the density functional theory. We found that the Fermi level of graphene is elevated by compressive strain and degraded by tensile strain. But uniaxial strain can give rise to the symmetry breaking of graphene and open the band gap. Furthermore, the tensile uniaxial strain is more beneficial to the band gap opening than the compressive uniaxial strain when the uniaxial strain is perpendicular to the C-C bond, but the compressive uniaxial strain is more than the tensile uniaxial strain when the uniaxial strain is parallel to the C-C bond. Second, the symmetry breaking of graphene resulting from uniaxial strain can be illustrated in that the uniaxial strain weakens the electron orbital coupling of graphene between px and py orbitals and brings about the splitting of the peak of the pz orbital density of states (DOS) on the left side of the Fermi level. Finally, whether uniaxial or biaxial strain, the compressive strain widens the pseudogap of graphene and the tensile strain narrows it. This would be useful for greatly broadening its applications in nanoelectronics and optoelectronics.

Embracing the Practical Aspects of Theranostics With Prostate-Specific Membrane Antigen-Targeted Lutetium-177.

Treatment options for men with metastatic castration-resistant prostate cancer are rapidly changing. In addition to novel anti-androgens and taxane-based chemotherapy, radiopharmaceuticals are having an increasing role. Although calcium-mimetic theranostics have been in use for years, newer approaches use molecularly targeted radiation therapy by conjugating isotopes to prostate-specific membrane antigen (PSMA) and in so doing directly target prostate cancer cells; 177Lutetium-PSMA-617 is perhaps the best-known member of this new class. Expanding our capacity to deliver targeted beta-emitters requires additional planning and equipment. Having delivered close to 200 doses of 177Lutetium-PSMA-617 at our center, we offer practical advice about patient selection, radiation safety, treatment administration, and toxicity monitoring. Although this blueprint is not the only way to expand a theranostics program beyond Radium-223, we offer our institutional experience with 177Lutetium-PSMA-617 as an example to programs seeking to expand their radiopharmaceutical programs. We must rise to meet the patient-driven demand for these innovative and effective therapies.

First-principles investigation of ScX2 (X = Cl, Br, or I) monolayers for flexible spintronic and electronic applications.

Two-dimensional (2D) nanomaterials with multiple characteristics are potential candidates as nanodevices for various applications. Herein, a family of 2D ScX2 (X = Cl, Br, or I) monolayers with functional properties are theoretically proposed to investigate their potential applications in spintronics and electronics. We show that these new nanomaterials possess ferromagnetic and semiconducting properties. The ScX2 monolayers hold intrinsic magnetism, which is ascribed to the ionic interaction between the Sc and X atoms. The magnetism is robust under repeated applied strains, which make them promising as flexible spintronic materials. Furthermore, we demonstrate that the ScX2 monolayers present semiconducting features and ultra-mobile carrier mobilities (∼103 cm2 V-1 s-1) benefited from their small deformation potentials. The carrier mobilities of the ScX2 monolayers are comparable with or more mobile than those of most 2D nanomaterials, which make them potential candidates as high-performance electronics. Our results suggest that the ScX2 monolayers may find various applications in flexible spintronics and electronics.

WXy /g-C3 N4 (WXy =W2 C, WS2 , or W2 N) Composites for Highly Efficient Photocatalytic Water Splitting.

The development of earth-abundant, economical, and efficient photocatalysts to boost water splitting is a key challenge for the practical large-scale application of hydrogen energy. In this study, g-C3 N4 loaded with different tungsten compounds (W2 C, WS2 , and W2 N) is found to exhibit enhanced photocatalytic activities. W2 C/g-C3 N4 displays the highest activity for the photocatalytic reaction with a H2 evolution rate of up to 98 µmol h-1 , as well as remarkable recycling stability. The excellent photocatalytic activity of W2 C/g-C4 N3 is attributed to the suitable band alignment in W2 C/g-C4 N3 and high HER activity of the W2 C cocatalyst, which promotes the separation and transfer of carriers and hydrogen evolution at the surface. These findings demonstrate that the tungsten carbide cocatalyst is more active for the photocatalytic reaction than the sulfide or nitride, paving a way for the design of novel and efficient carbides as cocatalysts for photocatalysis.

Two-Dimensional Janus Transition Metal Oxides and Chalcogenides: Multifunctional Properties for Photocatalysts, Electronics, and Energy Conversion.

The fast development of high-performance devices for diverse applications requires nanoscale materials with multifunctional properties, motivating theoretical exploration into novel two-dimensional (2D) materials. In this work, we propose a new family of 2D nanomaterials, Janus transition metal oxides and chalcogenides MXY (M = Ti, Zr, or Hf; X = S or Se; Y = O or S; X ≠ Y) monolayers, for their versatile applications. We find that the Janus MXY monolayers are semiconductors with a wide range of band gaps ranging from 0.739 to 2.884 eV. We show that TiSO, ZrSO, and HfSO monolayers are promising candidates for photocatalysis because of their suitable band gaps and optimal redox potentials for water splitting, and ZrSeS and HfSeS monolayers are suitable candidates for nanoscale electronics because of their high carrier mobility. We further show that TiSO, ZrSO, and ZrSeO monolayers possess large piezoelectric properties because of the broken inversion symmetry stemmed from the different atomic sizes and electronegativities of the X and Y elements, which are better or comparable to other 2D and bulk piezoelectric materials. Our study demonstrates that the 2D Janus MXYs may find versatile applications into photocatalysts, electronics, sensors, and energy harvesting/conversion.

Two-dimensional pentagonal CrX (X = S, Se or Te) monolayers: antiferromagnetic semiconductors for spintronics and photocatalysts.

Two dimensional (2D) materials with hexagonal building blocks have received tremendous interest in recent years and show promise as nanoscale devices for versatile applications. Herein, we propose a new family of 2D pentagonal CrX (X = S, Se or Te) monolayers (penta-CrX) for applications in electronics, spintronics and photocatalysis. We find that the 2D penta-CrX monolayers are thermally, structurally and mechanically stable. The penta-CrX monolayers are antiferromagnetic and semiconducting. We show that the magnetism is attributed to the super-exchange induced by the ionic interactions between the Cr and X atoms and can be enhanced upon applying tension. We further show that the penta-CrS and penta-CrSe monolayers show good redox potentials versus a normal hydrogen electrode, and their band gaps are comparable to the energy of a photon in the visible light region, indicating their capability of maximal utilization of solar energy for water splitting. With intrinsic semiconducting and controllable magnetic properties, the proposed penta-CrX monolayers may hold promise as flexible spintronics and photocatalysts.

Efficient nitrogen fixation to ammonia on MXenes.

Active catalysts for nitrogen fixation (N2-fixation) have been widely pursued through constant efforts for industrial applications. Here, we report a family of catalysts, MXenes (M2X: M = Mo, Ta, Ti, and W; X = C and N), for application in N2-fixation based on density functional theory calculations. We find that the catalytic performance of MXenes strongly depends on the reaction energy in each reaction step. More exothermic steps lead to higher catalytic performance in the course of N2-fixation. We show that the reaction energy in N2-fixation is strongly affected by the charge transfer: (1) if N atoms gain more electrons in a step, the reaction is exothermic with a larger reaction energy; (2) if N atoms lose electrons in a step, the reaction is endothermic in general. We further show that Mo2C and W2C are highly active for N2-fixation due to their exothermic reactions and strong charge transfer, which may be applicable in the chemical-engineering industry.

Establishing the feasibility of the dosimetric compliance criteria of RTOG 1308: phase III randomized trial comparing overall survival after photon versus proton radiochemotherapy for inoperable stage II-IIIB NSCLC.

BACKGROUND: To establish the feasibility of the dosimetric compliance criteria of the RTOG 1308 trial through testing against Intensity Modulation Radiation Therapy (IMRT) and Passive Scattering Proton Therapy (PSPT) plans. METHODS: Twenty-six lung IMRT and 26 proton PSPT plans were included in the study. Dose Volume Histograms (DVHs) for targets and normal structures were analyzed. The quality of IMRT plans was assessed using a knowledge-based engineering tool. RESULTS: Most of the RTOG 1308 dosimetric criteria were achieved. The deviation unacceptable rates were less than 10 % for most criteria; however, a deviation unacceptable rate of more than 20 % was computed for the planning target volume minimum dose compliance criterion. Dose parameters for the target volume were very close for the IMRT and PSPT plans. However, the PSPT plans led to lower dose values for normal structures. The dose parameters in which PSPT plans resulted in lower values than IMRT plans were: lung V5Gy (%) (34.4 in PSPT and 47.2 in IMRT); maximum spinal cord dose (31.7 Gy in PSPT and 43.5 Gy in IMRT); heart V5Gy (%) (19 in PSPT and 47 in IMRT); heart V30Gy (%) (11 in PSPT and 19 in IMRT); heart V45Gy (%) (7.8 in PSPT and 12.1 in IMRT); heart V50% (Gy) (7.1 in PSPT and 9.8 in IMRT) and mean heart dose (7.7 Gy in PSPT and 14.9 Gy in IMRT). CONCLUSIONS: The revised RTOG 1308 dosimetric compliance criteria are feasible and achievable.

Contouring variations and the role of atlas in non-small cell lung cancer radiation therapy: Analysis of a multi-institutional preclinical trial planning study.

PURPOSE: To quantify variations in target and normal structure contouring and evaluate dosimetric impact of these variations in non-small cell lung cancer (NSCLC) cases. To study whether providing an atlas can reduce potential variation. METHODS AND MATERIALS: Three NSCLC cases were distributed sequentially to multiple institutions for contouring and radiation therapy planning. No segmentation atlas was provided for the first 2 cases (Case 1 and Case 2). Contours were collected from submitted plans and consensus contour sets were generated. The volume variation among institution contours and the deviation of them from consensus contours were analyzed. The dose-volume histograms for individual institution plans were recalculated using consensus contours to quantify the dosimetric changes. An atlas containing targets and critical structures was constructed and was made available when the third case (Case 3) was distributed for planning. The contouring variability in the submitted plans of Case 3 was compared with that in first 2 cases. RESULTS: Planning target volume (PTV) showed large variation among institutions. The PTV coverage in institutions' plans decreased dramatically when reevaluated using the consensus PTV contour. The PTV contouring consistency did not show improvement with atlas use in Case 3. For normal structures, lung contours presented very good agreement, while the brachial plexus showed the largest variation. The consistency of esophagus and heart contouring improved significantly (t test; P < .05) in Case 3. Major factors contributing to the contouring variation were identified through a survey questionnaire. CONCLUSIONS: The amount of contouring variations in NSCLC cases was presented. Its impact on dosimetric parameters can be significant. The segmentation atlas improved the contour agreement for esophagus and heart, but not for the PTV in this study. Quality assurance of contouring is essential for a successful multi-institutional clinical trial.

Impact of dose calculation algorithm on radiation therapy.

The quality of radiation therapy depends on the ability to maximize the tumor control probability while minimize the normal tissue complication probability. Both of these two quantities are directly related to the accuracy of dose distributions calculated by treatment planning systems. The commonly used dose calculation algorithms in the treatment planning systems are reviewed in this work. The accuracy comparisons among these algorithms are illustrated by summarizing the highly cited research papers on this topic. Further, the correlation between the algorithms and tumor control probability/normal tissue complication probability values are manifested by several recent studies from different groups. All the cases demonstrate that dose calculation algorithms play a vital role in radiation therapy.

A semi-automated tool for treatment plan-quality evaluation and clinical trial quality assurance.

The goal of this work is to develop a plan-quality evaluation program for clinical routine and multi-institutional clinical trials so that the overall evaluation efficiency is improved. In multi-institutional clinical trials evaluating the plan quality is a time-consuming and labor-intensive process. In this note, we present a semi-automated plan-quality evaluation program which combines MIMVista, Java/MATLAB, and extensible markup language (XML). More specifically, MIMVista is used for data visualization; Java and its powerful function library are implemented for calculating dosimetry parameters; and to improve the clarity of the index definitions, XML is applied. The accuracy and the efficiency of the program were evaluated by comparing the results of the program with the manually recorded results in two RTOG trials. A slight difference of about 0.2% in volume or 0.6 Gy in dose between the semi-automated program and manual recording was observed. According to the criteria of indices, there are minimal differences between the two methods. The evaluation time is reduced from 10-20 min to 2 min by applying the semi-automated plan-quality evaluation program.

Implementation of remote 3-dimensional image guided radiation therapy quality assurance for radiation therapy oncology group clinical trials.

PURPOSE: To report the process and initial experience of remote credentialing of three-dimensional (3D) image guided radiation therapy (IGRT) as part of the quality assurance (QA) of submitted data for Radiation Therapy Oncology Group (RTOG) clinical trials; and to identify major issues resulting from this process and analyze the review results on patient positioning shifts. METHODS AND MATERIALS: Image guided radiation therapy datasets including in-room positioning CT scans and daily shifts applied were submitted through the Image Guided Therapy QA Center from institutions for the IGRT credentialing process, as required by various RTOG trials. A centralized virtual environment is established at the RTOG Core Laboratory, containing analysis tools and database infrastructure for remote review by the Physics Principal Investigators of each protocol. The appropriateness of IGRT technique and volumetric image registration accuracy were evaluated. Registration accuracy was verified by repeat registration with a third-party registration software system. With the accumulated review results, registration differences between those obtained by the Physics Principal Investigators and from the institutions were analyzed for different imaging sites, shift directions, and imaging modalities. RESULTS: The remote review process was successfully carried out for 87 3D cases (out of 137 total cases, including 2-dimensional and 3D) during 2010. Frequent errors in submitted IGRT data and challenges in the review of image registration for some special cases were identified. Workarounds for these issues were developed. The average differences of registration results between reviewers and institutions ranged between 2 mm and 3 mm. Large discrepancies in the superior-inferior direction were found for megavoltage CT cases, owing to low spatial resolution in this direction for most megavoltage CT cases. CONCLUSION: This first experience indicated that remote review for 3D IGRT as part of QA for RTOG clinical trials is feasible and effective. The magnitude of registration discrepancy between institution and reviewer was presented, and the major issues were investigated to further improve this remote evaluation process.

Application of Dempster-Shafer theory in dose response outcome analysis.

The Quantitative Analysis of Normal Tissue Effects in the Clinic (QUANTEC) reviews summarize the currently available three-dimensional dose/volume/outcome data from multi-institutions and numerous articles to update and refine the normal tissue dose/volume tolerance guidelines. As pointed out in the review, the data have limitations and even some inconsistency. However, with the help of new physical and statistical techniques, the information in the review could be updated so that patient care can be continually improved. The purpose of this work is to demonstrate the application of a mathematical theory, the Dempster-Shafer theory, in dose/volume/outcome data analysis. We applied this theory to the original data obtained from published clinical studies describing dose response for radiation pneumonitis. Belief and plausibility concepts were introduced for dose response evaluation. We were also able to consider the uncertainty and inconsistency of the data from these studies with Yager's combination rule, a special methodology of Dempster-Shafer theory, to fuse the data at several specific doses. The values of belief and plausibility functions were obtained at the corresponding doses. Then we applied the Lyman-Kutcher-Burman (LKB) model to fit these values and a belief-plausibility range was obtained. This range could be considered as a probability range to assist physicians and treatment planners in determining acceptable dose-volume constraints. Finally, the parameters obtained from the LKB model fitting were compared with those in Emami and Burman's papers and those from other frequentist statistics methods. We found that Emami and Burman's parameters are within the belief-plausibility range we calculated by the Dempster-Shafer theory.

Epidemiologic characterization of human papillomavirus infection in rural Chaozhou, eastern Guangdong Province of China.

BACKGROUND: Human papilloma virus (HPV) infection was the main cause of cervical cancer. There were only a few reports and detailed data about epidemiological research of HPV infection in rural population of China. MATERIALS AND METHODS: The cervical cells of rural Chaozhou women were collected, and multiplex real time PCR was firstly performed to detect high-risk HPV (HR-HPV) infection, which could detect 13 types of HR-HPV (types 16, 18, 31, 33, 35, 39, 45, 51, 52, 56, 58, 59, and 68). Then, HPV-positive samples were typed by HPV GenoArray test. RESULTS: HR-HPV DNA was detected by multiplex real time-PCR in 3830 of 48559 cases (7.89%). There was a peak incidence in age of 55-60 years group, and a lower incidence in who lived in plain group compared with suburban, mountain and seashore group. 3380 cases of HPV positive sample were genotyped, 11.01% (372/3380) cases could not be classified, among the typed 3008 cases, 101 cases were identified without HR-HPV type infection, 2907 cases were infected with one HR-HPV type at least, the 6 most common HR-HPV types in descending order of infection, were type 52 (33.4%, 16 (20.95%), 58 (15.93%), 33 (9.94%), 68 (9.22%) and 18 (8.36%). The combined prevalence of HPV types 16 and 18 accounted for 28.52% of total infection. However, type 52 plus 58 presented 48.23% of total infection. 2209/2907 cases were infected with a single HPV type and 698/2907 cases were infected with multiple types, and multiple infection constituent ratio increased with age, with a peak incidence in age 55-60 years group. CONCLUSIONS: Our findings showed low prevalence of HPV vaccine types (16 and 18) and relatively high prevalence of HPV-52 and -58, support the hypothesis that the second-generation HPV vaccines including HPV-52 and -58 may offer higher protection for women in rural Guangdong Province.

Characteristics of Gafchromic XRQA2 films for kV image dose measurement.

PURPOSE: In this study, the relevant characteristics of the new Gafchromic XRQA2 film for its application in measuring kV cone beam computed tomography (CBCT) image doses were thoroughly investigated. METHODS: The film was calibrated free in air to air kerma levels between 0 and 9 cGy using 120 kVp photon beams produced by the x-ray volume imager. Films were scanned using transmission and reflection scanning modes with the Epson Expression 10000 XL flat-bed document scanner. The impact of film size, region of interest for the analysis, scan uniformity, scan resolution, scan orientation and alternate scanning sides on the analysis process were investigated. Energy dependence, postirradiation growth of reflectance with time and irradiation angular dependence of the film were tested at different air kerma levels. RESULTS: The net reflectance changed by ∼3% when the size of the film piece changed from 1 cm × 2 cm to 10 cm × 11 cm and changed by ∼1% when ROI changed from 0. 7 cm × 0. 7 cm to 8 cm × 8 cm, suggesting a good uniformity of the film. The film was successfully analyzed using the transmission scanning mode, calibration curves from both transmission and reflection scanning modes showed similar behavior. The calibration uncertainty was somewhat lower when the film was scanned using reflection mode (6% and 8% for reflection and transmission modes, respectively.) Higher scanning resolution came with increasing calibration uncertainty. The calibration uncertainty for reflection and transmission modes increased from ∼3.5% to 7% and from ∼3.5% to 9%, respectively when scanning resolution was changed from 50 to 400 dpi. Scanning the film on alternate sides using transmission mode led to variation of 16%-19% in the net optical density at doses commonly used for CBCT procedures. The film response changed by almost 10% when it was exposed to beams of two different energies (100 and 120 kVp.) Other features of the film such as film orientation, postexposure growth, and irradiation angular dependence were also investigated. CONCLUSIONS: The size of film piece and analysis ROI used for calibration slightly affected the film response. Both transmission and reflection scanning modes can be used to analyze the Gafchromic XRQA2, with the reflection mode having a somewhat lower calibration uncertainty. Scanning films on alternate sides using transmission mode significantly affects the optical density. The film response was shown to be energy dependent. The films reached stability in about 6 h after exposure. The film response was proven to be independent of irradiation angle except when the beam is parallel to the film surface.