Real-Time Seamless Multi-Projector Displays on Deformable Surfaces.

Prior works on multi-projector displays have focused primarily on static rigid objects, some focusing on dynamic rigid objects. However, works on projection based displays on deformable dynamic objects have focused only on small scale single projector displays. Tracking a deformable dynamic surface and updating projections precisely in real time on it is a significantly challenging task, even for a single projector system. In this paper, we present the first end-to-end solution for achieving a real-time, seamless display on deformable surfaces using mutliple unsychronized projectors without requiring any prior knowledge of the surface or device parameters. The system first accurately calibrates multiple RGB-D cameras and projectors using the deformable display surface itself, and then using those calibrated devices, tracks the continuous changes in the surface shape. Based on the deformation and projector calibration, the system warps and blends the image content in real-time to create a seamless display on a surface that continuously changes shape. Using multiple projectors and RGB-D cameras, we provide the much desired aspect of scale to the displays on deformable surfaces. Most prior dynamic multi-projector systems assume rigid objects and depend critically on the constancy of surface normals and non-existence of local shape deformations. These assumptions break in deformable surfaces making prior techniques inapplicable. Point-based correspondences become inadequate for calibration, exacerbated with no synchronization between the projectors. A few works address non-rigid objects with several restrictions like targeting semi-deformable surfaces (e.g. human face), or using single coaxial (optically aligned) projector-camera pairs, or temporally synchronized cameras. We break loose from such restrictions and handle multiple projector systems for dynamic deformable fabric-like objects using temporally unsynchronized devices. We devise novel methods using ray and plane-based constraints imposed by the pinhole camera model to address these issues and design new blending methods dependent on 3D distances suitable for deformable surfaces. Finally, unlike all prior work with rigid dynamic surfaces that use a single RGB-D camera, we devise a method that involve all RGB-D cameras for tracking since the surface is not seen completely by a single camera. These methods enable a seamless display at scale in the presence of continuous movements and deformations. This work has tremendous applications on mobile and expeditionary systems where environmentals (e.g. wind, vibrations, suction) cannot be avoided. One can create large displays on tent walls in remote, austere military or emergency operations in minutes to support large scale command and control, mission rehearsal or training operations. It can be used to create displays on mobile and inflatable objects for tradeshows/events and touring edutainment applications.

3D Gamut Morphing for Non-Rectangular Multi-Projector Displays.

In a spatially augmented reality system, multiple projectors are tiled on a complex shaped surface to create a seamless display on it. This has several applications in visualization, gaming, education and entertainment. The main challenges in creating seamless and undistorted imagery on such complex shaped surfaces are geometric registration and color correction. Prior methods that provide solutions for the spatial color variation in multi-projector displays assume rectangular overlap regions across the projectors that is possible only on flat surfaces with extremely constrained projector placement. In this paper, we present a novel and fully automated method for removing color variations in a multi-projector display on arbitrary shaped smooth surfaces using a general color gamut morphing algorithm that can handle any arbitrarily shaped overlap between the projectors and assures imperceptible color variations across the display surface.

Generation of anti-GD2 CAR macrophages from human pluripotent stem cells for cancer immunotherapies.

Macrophages armed with chimeric antigen receptors (CARs) provide a potent new option for treating solid tumors. However, genetic engineering and scalable production of somatic macrophages remains significant challenges. Here, we used CRISPR-Cas9 gene editing methods to integrate an anti-GD2 CAR into the AAVS1 locus of human pluripotent stem cells (hPSCs). We then established a serum- and feeder-free differentiation protocol for generating CAR macrophages (CAR-Ms) through arterial endothelial-to-hematopoietic transition (EHT). CAR-M produced by this method displayed a potent cytotoxic activity against GD2-expressing neuroblastoma and melanoma in vitro and neuroblastoma in vivo. This study provides a new platform for the efficient generation of off-the-shelf CAR-Ms for antitumor immunotherapy.

Automated Geometric Registration for Multi-Projector Displays on Arbitrary 3D Shapes Using Uncalibrated Devices.

In this article we present a completely automated and scalable multi-projector registration system that allows multiple uncalibrated projectors and cameras on arbitrary shape surfaces. Our method estimates the parameters of multiple uncalibrated tiled or superimposed projectors, the extrinsic parameters of the observing cameras (with known intrinsic parameters), the shape of the illuminated 3D geometry and geometrically registers the projectors on it. This is achieved without using any fiducials, even if part of the surface is visible to only one camera. The method uses a completely automatic approach for cross-correlation and cross-validation of the device parameters and the surface geometry resulting in an accurate registration on the arbitrary unknown geometry that does not need an accurate prior calibration of each of the uncalibrated devices using physical patterns or fiducials. Estimating projector parameters allows for quick recalibration of the system in the face of projector movements, by re-estimating only the parameters of the moved projector and not the entire system. Thus, our work can enable easy deployment of spatially augmented reality environments of different sizes (from small table top objects to large immersive environments), different shapes (inside-looking-out or outside-looking in), and different configurations (tiled or superimposed) using the same proposed method.

Megakaryocytic Expansion in Gilteritinib-Treated Acute Myeloid Leukemia Patients Is Associated With AXL Inhibition.

Numerous recurrent genetic mutations are known to occur in acute myeloid leukemia (AML). Among these common mutations, Fms-like tyrosine kinase 3 remains as one of the most frequently mutated genes in AML. We observed apparent marrow expansion of megakaryocytes in three out of six patients with Flt3-mutated AML following treatment with a recently FDA-approved Flt3 inhibitor, gilteritinib which possesses activity against internal tandem duplication and tyrosine kinase domain Flt3 mutations and also inhibits tyrosine kinase AXL. To assess whether biopsy findings can be attributed to promotion of megakaryocytic (Mk) differentiation with gilteritinib, we devised a cellular assay by overexpressing double mutated Flt3-ITDY591F/Y919F in chronic myeloid leukemia cell line K562 to study Mk differentiation in the presence of Flt3 and AXL inhibitors with non-mutually exclusive mechanisms. These experiments demonstrated the lack of direct effect Flt3 inhibitors gilteritinib and quizartinib on megakaryocytic differentiation at either transcriptional or phenotypic levels, and highlighted antileukemic effects of AXL receptor tyrosine kinase inhibitor and its potential role in megakaryocytic development.

Generation of Human Neutrophils from Induced Pluripotent Stem Cells in Chemically Defined Conditions Using ETV2 Modified mRNA.

This protocol describes a rapid and efficient feeder-, serum-, and xeno-free method for neutrophil generation from hiPSCs using ETV2 modified mRNA (mmRNA), which directs hematoendothelial programming of hiPSCs. Hematoendothelial progenitors were cultured with GM-CSF, FGF-2, and UM171 to expand myelomonocytic progenitors, followed by treatment with G-CSF and retinoic acid agonist Am580 to induce neutrophil maturation. This protocol is suitable for generating functional neutrophils from iPSCs to interrogate the role of genes in a neutrophil development and function. For complete details on the use and execution of this protocol, please refer to Brok-Volchanskaya et al. (2019).

Histone chaperone HIRA dictate proliferation vs differentiation of chronic myeloid leukemia cells.

Abnormal proliferation and disrupted differentiation of hematopoietic progenitors mark leukemia. Histone cell cycle regulator A (HIRA), a histone chaperone, regulates hemogenic to hematopoietic transition involved in normal hematopoiesis. But, its role remains unexplored in leukemia, a case of dysregulated hematopoiesis. Here, the Cancer Cell Line Encyclopedia database analysis showed enhanced HIRA mRNA expression in cells of hematopoietic and lymphoid origin with maximal expression in the chronic myeloid leukemia (CML) cell line, K562. This observation was further endorsed by the induced expression of HIRA in CML patient samples compared to healthy individuals and Acute Myeloid Leukemia patients. Downregulation of HIRA in K562 cells displayed cell cycle arrest, loss in proliferation, presence of polyploidy with significant increase in CD41+ population thereby limiting proliferation but inducing differentiation of leukemia cells to megakaryocyte fate. Induced megakaryocyte differentiation of mouse Hira-knockout hematopoietic progenitors in vivo further confirmed the in vitro findings in leukemia cells. Molecular analysis showed the involvement of MKL1/GATA2/H3.3 axis in dictating differentiation of CML cells to megakaryocytes. Thus, HIRA could be exploited for differentiation induction therapy in CML and in chronic pathological conditions involving low platelet counts.

Optimized Multi-Spectral Filter Array Based Imaging of Natural Scenes.

Multi-spectral imaging using a camera with more than three channels is an efficient method to acquire and reconstruct spectral data and is used extensively in tasks like object recognition, relighted rendering, and color constancy. Recently developed methods are used to only guide content-dependent filter selection where the set of spectral reflectances to be recovered are known a priori. We present the first content-independent spectral imaging pipeline that allows optimal selection of multiple channels. We also present algorithms for optimal placement of the channels in the color filter array yielding an efficient demosaicing order resulting in accurate spectral recovery of natural reflectance functions. These reflectance functions have the property that their power spectrum statistically exhibits a power-law behavior. Using this property, we propose power-law based error descriptors that are minimized to optimize the imaging pipeline. We extensively verify our models and optimizations using large sets of commercially available wide-band filters to demonstrate the greater accuracy and efficiency of our multi-spectral imaging pipeline over existing methods.

Enhanced expression of histone chaperone APLF associate with breast cancer.

DNA damage-specific histone chaperone Aprataxin PNK-like factor (APLF) regulates mesenchymal-to-epithelial transition (MET) during cellular reprogramming. We investigated the role of APLF in epithelial-to-mesenchymal transition (EMT) linked to breast cancer invasiveness and metastasis. Here, we show that a significant manifestation of APLF is present in tumor sections of patients with invasive ductal carcinoma when compared to their normal adjacent tissues. APLF was significantly induced in triple negative breast cancer (TNBC) cells, MDAMB-231, in comparison to invasive MCF7 or normal MCF10A breast cells and supported by studies on invasive breast carcinoma in The Cancer Genome Atlas (TCGA). Functionally, APLF downregulation inhibited proliferative capacity, altered cell cycle behavior, induced apoptosis and impaired DNA repair ability of MDAMB-231 cells. Reduction in APLF level impeded invasive, migratory, tumorigenic and metastatic potential of TNBC cells with loss in expression of genes associated with EMT while upregulation of MET-specific gene E-cadherin (CDH1). So, here we provided novel evidence for enrichment of APLF in breast tumors, which could regulate metastasis-associated EMT in invasive breast cancer. We anticipate that APLF could be exploited as a biomarker for breast tumors and additionally could be targeted in sensitizing cancer cells towards DNA damaging agents.

Histone chaperone APLF regulates induction of pluripotency in murine fibroblasts.

Induction of pluripotency in differentiated cells through the exogenous expression of the transcription factors Oct4, Sox2, Klf4 and cellular Myc involves reprogramming at the epigenetic level. Histones and their metabolism governed by histone chaperones constitute an important regulator of epigenetic control. We hypothesized that histone chaperones facilitate or inhibit the course of reprogramming. For the first time, we report here that the downregulation of histone chaperone Aprataxin PNK-like factor (APLF) promotes reprogramming by augmenting the expression of E-cadherin (Cdh1), which is implicated in the mesenchymal-to-epithelial transition (MET) involved in the generation of induced pluripotent stem cells (iPSCs) from mouse embryonic fibroblasts (MEFs). Downregulation of APLF in MEFs expedites the loss of the repressive MacroH2A.1 (encoded by H2afy) histone variant from the Cdh1 promoter and enhances the incorporation of active histone H3me2K4 marks at the promoters of the pluripotency genes Nanog and Klf4, thereby accelerating the process of cellular reprogramming and increasing the efficiency of iPSC generation. We demonstrate a new histone chaperone (APLF)-MET-histone modification cohort that functions in the induction of pluripotency in fibroblasts. This regulatory axis might provide new mechanistic insights into perspectives of epigenetic regulation involved in cancer metastasis.

Live detection and purification of cells based on the expression of a histone chaperone, HIRA, using a binding peptide.

Flow cytometry is a reliable method for identification and purification of live cells from a heterogeneous population. Since permeabilized cells cannot be sorted live in a FACS sorter, its application in isolation of functional cells largely depends on antibodies for surface markers. In various fields of biology we find intracellular markers that reveal subpopulations of biological significance. Cell cycle stage specific molecules, metastatic signature molecules, stemness associated proteins etc. are examples of potential markers that could improve the research and therapy enormously. Currently their use is restricted by lack of techniques that allow live detection. Even though a few methods like aptamers, droplet-based microfluidics and smartflares are reported, their application is limited. Here, for the first time we report a simple, cost-effective and efficient method of live sorting of cells based on the expression of an intracellular marker using a fluorophore-tagged binding peptide. The target molecule selected was a histone chaperone, HIRA, the expression of which can predict the fate of differentiating myoblast. Our results confirm that the peptide shows specific interaction with its target; and it can be used to separate cells with differential expression of HIRA. Further, this method offers high purity and viability for the isolated cells.

A Distributed Memory Hierarchy and Data Management for Interactive Scene Navigation and Modification on Tiled Display Walls.

Simultaneous modification and navigation of massive 3D models are difficult because repeated data edits affect the data layout and coherency on a secondary storage, which in turn affect the interactive out-of-core rendering performance. In this paper, we propose a novel approach for distributed data management for simultaneous interactive navigation and modification of massive 3D data using the readily available infrastructure of a tiled display. Tiled multi-displays, projection or LCD panel based, driven by a PC cluster, can be viewed as a cluster of storage-compute-display (SCD) nodes. Given a cluster of SCD node infrastructure, we first propose a distributed memory hierarchy for interactive rendering applications. Second, in order to further reduce the latency in such applications, we propose a new data partitioning approach for distributed storage among the SCD nodes that reduces the variance in the data load across the SCD nodes. Our data distribution method takes in a data set of any size, and reorganizes it into smaller partitions, and stores it across the multiple SCD nodes. These nodes store, manage, and coordinate data with other SCD nodes to simultaneously achieve interactive navigation and modification. Specifically, the data is not duplicated across these distributed secondary storage devices. In addition, coherency in data access, due to screen-space adjacency of adjacent displays in the tile, as well as object space adjacency of the data sets, is well leveraged in the design of the data management technique. Empirical evaluation on two large data sets, with different data density distribution, demonstrates that the proposed data management approach achieves superior performance over alternative state-of-the-art methods.

Histone Chaperone HIRA in Regulation of Transcription Factor RUNX1.

RUNX1 (Runt-related transcription factor 1) is indispensable for the generation of hemogenic endothelium. However, the regulation of RUNX1 during this developmental process is poorly understood. We investigated the role of the histone chaperone HIRA (histone cell cycle regulation-defective homolog A) from this perspective and report that HIRA significantly contributes toward the regulation of RUNX1 in the transition of differentiating mouse embryonic stem cells from hemogenic to hematopoietic stage. Direct interaction of HIRA and RUNX1 activates the downstream targets of RUNX1 implicated in generation of hematopoietic stem cells. At the molecular level, HIRA-mediated incorporation of histone H3.3 variant within the Runx1 +24 mouse conserved noncoding element is essential for the expression of Runx1 during endothelial to hematopoietic transition. An inactive chromatin at the intronic enhancer of Runx1 in absence of HIRA significantly repressed the transition of cells from hemogenic to hematopoietic fate. We expect that the HIRA-RUNX1 axis might open up a novel approach in understanding leukemogenesis in future.

Using Patterns to Encode Color Information for Dichromats.

Color is one of the most common ways to convey information in visualization applications. Color vision deficiency (CVD) affects approximately 200 million individuals worldwide and considerably degrades their performance in understanding such contents by creating red-green or blue-yellow ambiguities. While several content-specific methods have been proposed to resolve these ambiguities, they cannot achieve this effectively in many situations for contents with a large variety of colors. More importantly, they cannot facilitate color identification. We propose a technique for using patterns to encode color information for individuals with CVD, in particular for dichromats. We present the first content-independent method to overlay patterns on colored visualization contents that not only minimizes ambiguities but also allows color identification. Further, since overlaying patterns does not compromise the underlying original colors, it does not hamper the perception of normal trichromats. We validated our method with two user studies: one including 11 subjects with CVD and 19 normal trichromats, and focused on images that use colors to represent multiple categories; and another one including 16 subjects with CVD and 22 normal trichromats, which considered a broader set of images. Our results show that overlaying patterns significantly improves the performance of dichromats in several color-based visualization tasks, making their performance almost similar to normal trichromats'. More interestingly, the patterns augment color information in a positive manner, allowing normal trichromats to perform with greater accuracy.

Autocalibration of multiprojector CAVE-like immersive environments.

In this paper, we present the first method for the geometric autocalibration of multiple projectors on a set of CAVE-like immersive display surfaces including truncated domes and 4 or 5-wall CAVEs (three side walls, floor, and/or ceiling). All such surfaces can be categorized as swept surfaces and multiple projectors can be registered on them using a single uncalibrated camera without using any physical markers on the surface. Our method can also handle nonlinear distortion in the projectors, common in compact setups where a short throw lens is mounted on each projector. Further, when the whole swept surface is not visible from a single camera view, we can register the projectors using multiple pan and tilted views of the same camera. Thus, our method scales well with different size and resolution of the display. Since we recover the 3D shape of the display, we can achieve registration that is correct from any arbitrary viewpoint appropriate for head-tracked single-user virtual reality systems. We can also achieve wallpapered registration, more appropriate for multiuser collaborative explorations. Though much more immersive than common surfaces like planes and cylinders, general swept surfaces are used today only for niche display environments. Even the more popular 4 or 5-wall CAVE is treated as a piecewise planar surface for calibration purposes and hence projectors are not allowed to be overlapped across the corners. Our method opens up the possibility of using such swept surfaces to create more immersive VR systems without compromising the simplicity of having a completely automatic calibration technique. Such calibration allows completely arbitrary positioning of the projectors in a 5-wall CAVE, without respecting the corners.

Autocalibrating Tiled Projectors on Piecewise Smooth Vertically Extruded Surfaces.

In this paper, we present a novel technique to calibrate multiple casually aligned projectors on fiducial-free piecewise smooth vertically extruded surfaces using a single camera. Such surfaces include cylindrical displays and CAVEs, common in immersive virtual reality systems. We impose two priors to the display surface. We assume the surface is a piecewise smooth vertically extruded surface for which the aspect ratio of the rectangle formed by the four corners of the surface is known and the boundary is visible and segmentable. Using these priors, we can estimate the display's 3D geometry and camera extrinsic parameters using a nonlinear optimization technique from a single image without any explicit display to camera correspondences. Using the estimated camera and display properties, the intrinsic and extrinsic parameters of each projector are recovered using a single projected pattern seen by the camera. This in turn is used to register the images on the display from any arbitrary viewpoint making it appropriate for virtual reality systems. The fast convergence and robustness of this method is achieved via a novel dimension reduction technique for camera parameter estimation and a novel deterministic technique for projector property estimation. This simplicity, efficiency, and robustness of our method enable several coveted features for nonplanar projection-based displays. First, it allows fast recalibration in the face of projector, display or camera movements and even change in display shape. Second, this opens up, for the first time, the possibility of allowing multiple projectors to overlap on the corners of the CAVE-a popular immersive VR display system. Finally, this opens up the possibility of easily deploying multiprojector displays on aesthetic novel shapes for edutainment and digital signage applications.

Projector placement planning for high quality visualizations on real-world colored objects.

Many visualization applications benefit from displaying content on real-world objects rather than on a traditional display (e.g., a monitor). This type of visualization display is achieved by projecting precisely controlled illumination from multiple projectors onto the real-world colored objects. For such a task, the placement of the projectors is critical in assuring that the desired visualization is possible. Using ad hoc projector placement may cause some appearances to suffer from color shifting due to insufficient projector light radiance being exposed onto the physical surface. This leads to an incorrect appearance and ultimately to a false and potentially misleading visualization. In this paper, we present a framework to discover the optimal position and orientation of the projectors for such projection-based visualization displays. An optimal projector placement should be able to achieve the desired visualization with minimal projector light radiance. When determining optimal projector placement, object visibility, surface reflectance properties, and projector-surface distance and orientation need to be considered. We first formalize a theory for appearance editing image formation and construct a constrained linear system of equations that express when a desired novel appearance or visualization is possible given a geometric and surface reflectance model of the physical surface. Then, we show how to apply this constrained system in an adaptive search to efficiently discover the optimal projector placement which achieves the desired appearance. Constraints can be imposed on the maximum radiance allowed by the projectors and the projectors' placement to support specific goals of various visualization applications. We perform several real-world and simulated appearance edits and visualizations to demonstrate the improvement obtained by our discovered projector placement over ad hoc projector placement.