When Pigs Fly: Contextual Reasoning in Synthetic and Natural Scenes.

Context is of fundamental importance to both human and machine vision; e.g., an object in the air is more likely to be an airplane than a pig. The rich notion of context incorporates several aspects including physics rules, statistical co-occurrences, and relative object sizes, among others. While previous work has focused on crowd-sourced out-of-context photographs from the web to study scene context, controlling the nature and extent of contextual violations has been a daunting task. Here we introduce a diverse, synthetic Out-of-Context Dataset (OCD) with fine-grained control over scene context. By leveraging a 3D simulation engine, we systematically control the gravity, object co-occurrences and relative sizes across 36 object categories in a virtual household environment. We conducted a series of experiments to gain insights into the impact of contextual cues on both human and machine vision using OCD. We conducted psychophysics experiments to establish a human benchmark for out-of-context recognition, and then compared it with state-of-the-art computer vision models to quantify the gap between the two. We propose a context-aware recognition transformer model, fusing object and contextual information via multi-head attention. Our model captures useful information for contextual reasoning, enabling human-level performance and better robustness in out-of-context conditions compared to baseline models across OCD and other out-of-context datasets. All source code and data are publicly available at https://github.com/kreimanlab/WhenPigsFlyContext.

Putting visual object recognition in context.

Context plays an important role in visual recognition. Recent studies have shown that visual recognition networks can be fooled by placing objects in inconsistent contexts (e.g. a cow in the ocean). To understand and model the role of contextual information in visual recognition, we systematically and quantitatively investigated ten critical properties of where, when, and how context modulates recognition including amount of context, context and object resolution, geometrical structure of context, context congruence, time required to incorporate contextual information, and temporal dynamics of contextual modulation. The tasks involve recognizing a target object surrounded with context in a natural image. As an essential benchmark, we first describe a series of psychophysics experiments, where we alter one aspect of context at a time, and quantify human recognition accuracy. To computationally assess performance on the same tasks, we propose a biologically inspired context aware object recognition model consisting of a two-stream architecture. The model processes visual information at the fovea and periphery in parallel, dynamically incorporates both object and contextual information, and sequentially reasons about the class label for the target object. Across a wide range of behavioral tasks, the model approximates human level performance without retraining for each task, captures the dependence of context enhancement on image properties, and provides initial steps towards integrating scene and object information for visual recognition.

The location and nature of general anesthetic binding sites on the active conformation of firefly luciferase; a time resolved photolabeling study.

Firefly luciferase is one of the few soluble proteins that is acted upon by a wide variety of general anesthetics and alcohols; they inhibit the ATP-driven production of light. We have used time-resolved photolabeling to locate the binding sites of alcohols during the initial light output, some 200 ms after adding ATP. The photolabel 3-azioctanol inhibited the initial light output with an IC50 of 200 µM, close to its general anesthetic potency. Photoincorporation of [(3)H]3-azioctanol into luciferase was saturable but weak. It was enhanced 200 ms after adding ATP but was negligible minutes later. Sequencing of tryptic digests by HPLC-MSMS revealed a similar conformation-dependence for photoincorporation of 3-azioctanol into Glu-313, a residue that lines the bottom of a deep cleft (vestibule) whose outer end binds luciferin. An aromatic diazirine analog of benzyl alcohol with broader side chain reactivity reported two sites. First, it photolabeled two residues in the vestibule, Ser-286 and Ile-288, both of which are implicated with Glu-313 in the conformation change accompanying activation. Second, it photolabeled two residues that contact luciferin, Ser-316 and Ser-349. Thus, time resolved photolabeling supports two mechanisms of action. First, an allosteric one, in which anesthetics bind in the vestibule displacing water molecules that are thought to be involved in light output. Second, a competitive one, in which anesthetics bind isosterically with luciferin. This work provides structural evidence that supports the competitive and allosteric actions previously characterized by kinetic studies.

DpgC is a metal- and cofactor-free 3,5-dihydroxyphenylacetyl-CoA 1,2-dioxygenase in the vancomycin biosynthetic pathway.

3,5-Dihydroxyphenylglycine is a crucial amino acid monomer in the nonribosomal glycopeptide antibiotic vancomycin. This nonproteinogenic amino acid is constructed from malonyl-CoA by a set of four enzymes, DpgA-D, in the biosynthetic cluster. DpgC is an unusual metal-free, cofactor-free enzyme that consumes O(2) during the conversion of 3,5-dihydroxyphenylacetyl-CoA (DPA-CoA) to the penultimate intermediate 3,5-dihydroxyphenylglyoxylate (DPGx). We show that in anaerobic incubations, DpgC catalyzes the exchange of the C(2)-methylene hydrogens of DPA-CoA at unequal rates, consistent with enzyme-mediated formation of the substrate-derived C(2)-carbanion as an early intermediate. Incubations with (18)O(2) reveal that DpgC transfers both atoms of an O(2) molecule to DPGx product. This establishes DpgC as a 1,2-dioxygenase that mediates thioester cleavage by the oxygen transfer process. These results are consistent with a DPA-CoA C(2)-peroxy intermediate, followed by enzyme-directed alpha-peroxylactone formation and collapse by O-O bond cleavage.

Role of the active site cysteine of DpgA, a bacterial type III polyketide synthase.

DpgA is a bacterial type III polyketide synthase (PKS) that decarboxylates and condenses four malonyl-CoA molecules to produce 3,5-dihydroxyphenylacetyl-CoA (DPA-CoA) in the biosynthetic pathway to 3,5-dihydroxyphenylglycine, a key nonproteinogenic residue in the vancomycin family of antibiotics. DpgA has the conserved catalytic triad of Cys/His/Asn typical of type III PKS enzymes, and has been assumed to use Cys160 as the catalytic nucleophile to create a series of elongating acyl-S-enzyme intermediates prior to the C(8) to C(3) cyclization step. Incubation of purified DpgA with [(14)C]-malonyl-CoA followed by acid quench during turnover leads to accumulation of 10-15% of the DpgA molecules covalently acylated. Mutation of the active site Cys160 to Ala abrogated detectable covalent acylation, but the C160A mutant retained 50% of the V(max) for DPA-CoA formation, with a k(cat) still at 0.5 catalytic turnovers/min. For comparison, a C190A mutant retained wild-type activity, while the H296A mutant, in which the side chain of the presumed catalytic His is removed, had a 6-fold drop in k(cat). During turnover, purified DpgA produced 1.2 equivalents of acetyl-CoA for each DPA-CoA, indicating 23% uncoupled decarboxylation competing with condensative C-C coupling. The C160A mutant showed an increased partition ratio for malonyl-CoA decarboxylation to acetyl-CoA vs condensation to DPA-CoA, reflecting more uncoupling in the mutant enzyme. The Cys-to-Ala mutant thus shows the unexpected result that, when the normal acyl-S-enzyme mechanism for this type III PKS elongation/cyclization catalyst is removed, it can still carry out the regioselective construction of the eight-carbon DPA-CoA skeleton with surprising efficiency.

Characterization of the surfactin synthetase C-terminal thioesterase domain as a cyclic depsipeptide synthase.

The C-terminal thioesterase domain of the nonribosomal peptide synthetase producing the lipopetide surfactin (Srf TE) retains autonomous ability to generate the cyclic peptidolactone skeleton of surfactin when provided with a soluble beta-hydroxy-butyryl-heptapeptidyl thioester substrate. Utilizing the recently solved crystal structure [Bruner, S. D., et al. (2002) Structure 10, 301-310], the active-site nucleophile, Ser80, was changed to Cys, and the other members of the catalytic triad, Asp107 and His207, were changed to Ala, with the resulting mutants lacking detectable activity. Two cationic side chains in the active site, Lys111 and Arg120, were changed to Ala, causing an increased partitioning of the product to hydrolysis, as did a P26G mutant, mimicking the behavior of lipases. To evaluate recognition elements in substrates used by Srf TE, alterations to the fatty acyl group, the heptapeptide, and the thioester leaving group were made, and the resulting substrates were characterized for kinetic competency and flux of product to cyclization or hydrolysis. Alterations that could be accepted for cyclization were identified in all three parts of the substrate, although tolerance limits for changes varied. In addition, cocrystal structures of Srf TE with dipeptidyl boronate inhibitors were solved, illustrating the critical binding determinants of the substrate. On the basis of the structures and biochemical data, the cyclizing conformation of the surfactin peptide was modeled into the enzyme active site.