UC Berkeley

Before a new nuclear reactor design can be constructed and operated, its safety must bedemonstrated using models that are validated with integral effects test (IET) data. However, because scaled integral effects tests are electrically heated, they do not exhibit nuclear reactor feedback phenomena. To replicate the nuclear transient response in electrically heated IETs, we require simulated neutronics feedback (SNF) controllers. Such SNF controllers can then be used to provide SNF capabilities for IET facilities such as the Compact Integral Effects Test (CIET) at the University of California, Berkeley (UC Berkeley). However, developing SNF controllers for IET facilities is non-trivial. To expedite development, we present the use of Digital Twins as testbeds for iterative SNF controller development. In particular, we use a Digital Twin of the Heater within CIET as a testbed for SNF Controller Development. This Digital Twin with SNF Capabilty is run as an OPC-UA server and client written almost entirely in Rust using Free and Open Source (FOSS) code. We then validate the Digital Twin with experimental data in literature. We also verify the transfer function simulation and Proportional, Integral and Derivative (PID) controllers written in Rust using Scilab. Moreover, we demonstrate use of data driven surrogate models (transfer functions) to construct SNF controllers in contrast to using the traditional Point Reactor Kinetics Equations (PRKE) models with the hope that they can account for the effect of spatially dependent neutron flux on reactor feedback. To construct the first surrogate models in this work, we use transient data from a representative arbitrary Fluoride Salt Cooled High Temperature Reactor (FHR) model constructed using OpenMC and GeN-Foam. Using the Digital Twin as a testbed, two design iterations of the SNF controller were developed using the data driven surrogate model. Compared to the potential development time taken in using physical experiments, using the digital twin testbed for SNF controller development resulted in a significant time saving. We hope that the approaches used in this dissertation can expedite testing and reduce expenditure for licensing novel Gen IV nuclear reactor designs.

The advent of advanced computational technologies and artificial intelligence has ushered in a new era of complex systems and applications, notably in the realms of autonomous vehicles (AVs) and robotics. These systems are increasingly required to make decisions autonomously in dynamic and uncertain environments. Reinforcement Learning (RL) has emerged as a pivotal technique in this context, offering a framework for learning optimal decision-making strategies through interactions with the environment. However, ensuring safety and trustworthiness in these decisions remains a critical challenge, especially in safety-critical applications such as autonomous driving.

This dissertation addresses the aforementioned challenge by proposing innovative RL-based approaches, and is structured into three distinct but interconnected parts, each focusing on a unique aspect of RL in the context of safe and trustworthy decision-making.The thread of this dissertation is based on the exploration and advancement of RL techniques to ensure safety and reliability in autonomous decision-making systems, particularly in complex, dynamic environments.

We first establish the foundational aspects of RL in decision-making, particularly in uncertain and dynamic environments. The focus here is on enhancing RL to deal with real-world complexities, such as interacting with unpredictable agents, e.g., human drivers in AV scenarios, and handling distributional shifts in offline RL settings. This sets the stage for understanding and improving the decision-making capabilities of autonomous systems under uncertainty.

Building on the first part, we then explore the integration of hierarchical planning with RL. The emphasis is on creating frameworks that combine different levels of decision-making, balancing immediate, low-level safety concerns with high-level strategic objectives. The approach aims to address the limitations of traditional RL in complex, multi-agent environments and long-duration tasks, demonstrating improved adaptability and efficiency in real-time decision-making.

The final part represents a forward-looking approach to RL, focusing on the integration of offline and online learning methodologies. This part addresses the challenge of training RL agents in a manner that is both safe and effective, particularly in contexts where exploration can be costly or dangerous. By combining the strengths of large-scale offline data (expert demonstrations) with online learning, we present a novel framework for enhancing the safety and performance of RL agents in practical, real-world applications.

A sequence of electron clouds is extracted from an electron plasma reservoir. These clouds are highly reproducible and their E x B drift motion is nearly identical to that of a single particle, making them useful for measurements of electric and magnetic fields. First, by weakening the trapping potential confining the clouds we observe that they move off-axis, and we use this to measure the electric field due to patch potentials. Next, we measure the total charge of these clouds using small shifts in their magnetron frequencies. The misalignment between the trap electrodes and the external magnet is measured by imaging the clouds from different axial locations in the trap. By combining electron cyclotron resonance with the patch potential measurement procedure, we can measure the magnetic field strength up to a millimeter away from the trap axis. Finally, a new magnetometry technique called electron magnetron phase imaging (EMPI) is used to measure the rapidly changing magnetic field involved in observing the effect of gravity on antihydrogen. In EMPI, the magnetron frequency is measured precisely, and then we observe small changes to the magnetron frequency as the magnetic field decreases. In the process of analyzing the experimental data from each of these measurements, subtleties in the motion of electron clouds are revealed. Some of these measurement techniques help us to understand systematic errors in the ALPHA collaboration's test of the weak equivalence principle. Other techniques are used to inform experimental procedures and help explain the behavior of ALPHA's Penning-Malmberg traps. Most of these ideas could be applied to many Penning-Malmberg traps, provided that they have the ability to image charged particles. Unknown magnetic fields, patch potentials, and misalignment pose difficulties for many experiments, so implementing these cloud-based measurements could benefit other research groups.

Chapter 1. The field of f-block–transition metal hydride chemistry is introduced and summarized. Key properties of these compounds such as small molecule activation chemistry and H2 uptake and release are outlined. The dearth of actinide–transition metal species despite their potential for fundamental bonding insight and novel reactivity is highlighted, and the motivations for studying these compounds are stated.

Chapter 2. Reaction of K[Cp*IrH3] with actinide halides led to multimetallic actinide–transition metal hydrides U{(μ-H)3IrCp*}4 and Th{[(μ-H)2(H)IrCp*]2[(μ-H)3IrCp*]2}, respectively. These complexes feature an unexpected, significant discrepancy in hydride bonding modes; the uranium species contains twelve bridging hydrides while the thorium complex contains ten bridging hydrides and two terminal, Ir-bound hydrides. Use of a U(III) starting material with the same potassium iridate resulted in the octanuclear complex {U[(μ2-H)3IrCp*]2[(μ3-H)2IrCp*]}2. Computational studies indicate significant bonding character between U/Th and Ir in the tetrairidate compounds, the first reported evidence of actinide-iridium covalency. In addition, these studies attribute the variation in hydride bonding between the tetrairidate complexes to differences in dispersion effects. This work establishes a novel route to synthesizing actinide–transition metal polyhydrides with close metal–metal contacts.

Chapter 3. Conversion of Cp*OsH5 to K[Cp*OsH4] with KBn, followed by reaction with tetravalent actinide halides results in the synthesis of uranium– and thorium–osmium heterometallic polyhydride complexes. Through these species, An–Os bonding and the reactivity of An–Os interactions are studied. These complexes are formally sixteen-coordinate, the highest observed coordination number for uranium and thorium. Computational studies suggest the presence of a significant bonding interaction between the actinide center and the four coordinated osmium centers, the first report of this behavior between osmium and an actinide. Upon photolysis, these complexes underwent intramolecular C–H activation with the formation of an Os–Os bond, while the thorium complex was able to activate an additional C–H bond of the benzene solvent, resulting in a μ-η1,η1 phenyl ligand across one Th–Os interaction. These results highlight the unique reactivity that can arise from actinide and transition metal centers in proximity, and expand the scope of actinide photolysis reactivity.

Chapter 4. The third Cp*-supported transition metal polyhydride – Cp*ReH6 – was shown to be a competent partner to actinide hydrides. The synthesis of actinide tetrarhenate complexes completed a series of iridate, osmate, and rhenate polyhydrides, allowing for structural and bonding comparisons. Computational studies examine the bonding interactions, particularly between metals, in these complexes. Several factors affect metal–metal distances and covalency for the actinide tetrametallates, including metal oxidation state, coordination number, and dispersion effects. The osmium and rhenium octametallic U2M6 clusters are reported as well, with similar analysis of structure and electronics.

Chapter 5. Reaction of the potassium iridate K[Cp*IrH3] with a bulky uranium(III) metallocene yielded a heterobimetallic U(III)–Ir species. Reactivity of this complex with CS2 is described, resulting in the novel ethanetetrathiolate fragment, as produced via hydride insertion and C–C coupling. This demonstrates the ability to combine the hydride insertion chemistry of transition metal hydrides with C–C coupling observed in U(III) compounds by bringing both metal centers in close proximity.

This dissertation leverages and develops the powerful out-of-sample safety certificates of Wasserstein ambiguity sets to create a suite of data-driven control algorithms that help solve safety-critical industrial problems. This work is motivated by the ongoing relevance of robustness and safety when applying data-driven decision making in the real world. For example, lithium-ion batteries are driving transitions to renewable energy sources. Optimizing their performance and longevity is of the utmost importance, but highly difficult due to their complex, nonlinear, and safety-critical electrochemical dynamics. While data-driven control can dramatically improve the performance of systems like lithium-ion batteries, certifying system safety remains an open challenge. This dissertation explores certifying learning-based controllers via distributionally robust optimization (DRO). We focus on Wasserstein ambiguity sets, DRO methods that draw worst-case realizations of random variables under relatively permissive assumptions. This makes them ideal for learning-based control, where data can be highly limited and the controller is likely encounter new experience unaccounted for in its training data.

In Chapter 2, we begin by presenting simple mathematical arguments that extend an existing reformulation of Wasserstein DRO to cases where dependence on decision variables x and random variables R can be nonconvex as long as x and R are separable. By cleverly modeling stochasticity in model uncertainty, we augment nonconvex optimal control problems with Wasserstein ambiguity sets to obtain idealized probabilistic safety certificates.

The remaining chapters extend this theoretical result across the range of model-based and model-free reinforcement learning. Chapter 2 explores offline model-based reinforcement learning within a latent state-space, with application to real-time fast-charging of li-ion batteries using electrochemical information. By leveraging the results of Chapter 2, we can hedge against model and data errors to probabilistically guarantee safe distributional data-driven control.

Chapter 4 presents an end-to-end framework for safe learning-based control using nonlinear stochastic MPC. We focus on scenarios where the controller is applied directly to a system of which it has highly limited experience, toward safety during tabula-rasa learning-based control as a challenging case for validation. We validate findings with case studies of extreme lithium-ion battery fast charging and autonomous vehicle obstacle avoidance using a basic perception system.

Finally, in Chapter 5, we apply the same DRO architecture to value-based RL. We describe a structure for deep Q-learning within the framework of constrained Markov decision processes (CMDPs). By characterizing the uncertainty of constraint cost functions based on their temporal-difference errors, we augment relevant constraints with tightening offset variables based on DRO theory of Chapter 2.

In our concluding remarks, we discuss the broader relevance of our findings and map directions for future work.

Random ordinary differential equations (RODEs) describe numerous physical and biological systems whose dynamics contain some level of inherent randomness. These sources of uncertainty enter into dynamics in two forms: (a) externally imposed or internally generated random excitations, i.e., noise, and/or (b) probabilistic representations of uncertain coefficients and initial/boundary data. Such systems admit a distribution of solutions, which is (partially) characterized by the single-time joint probability density function (PDF) of system states. If the random excitations correspond to Gaussian white noise, it is relatively straightforward to derive a closed-form deterministic partial differential equation (PDE) known as the Fokker-Planck (or Kolmogorov’s forward) equation, which governs the evolution of the joint PDF. However, most plausible noise sources are correlated (colored). In this case, the resulting PDF equations require a closure approximation. Via the method of distributions, we propose two methods for closing such equations: (a) modified large-eddy-diffusivity closures, and (b) a data-driven closure relying on sparse regression to learn relevant features. In the realms of nonequilibrium statistical mechanics and computational neuroscience, the closures are tested in a head-to-head comparison against Monte Carlo simulations for colored-noise sources such as Ornstein-Uhlenbeck and sine-Wiener processes. Additionally, the approaches’ algorithmic complexities are thoroughly discussed.

Implementing the method of distributions for high-dimensional systems of RODEs is challenging due to the computational burden of solving the high-dimensional PDE associated with the joint PDF of states. Although recent advancements in numerical integration techniques for high-dimensional PDEs have been made, they are often tailored to specific applications and lack generality for large numbers of states/dimensions. However, for many applications, only a low-dimensional quantity of interest (QoI) from the underlying high-dimensional system is desired. In these cases, it is sufficient to study a reduced-order PDF (RO-PDF) equation, i.e., a low-dimensional PDE for the QoI’s PDF, allowing classical integration techniques to be employed. Moreover, unclosed coefficients in the RO-PDF equations can be rewritten as conditional expectations, which we directly estimate from data via nonparametric regression. When the RODE exhibits strong nonlinearities and/or stiffness, it is usually necessary to supplement the learned reduced-order PDE with a data assimilation method to account for model misspecification that may occur from regression discrepancies. We propose nudging (a.k.a., Newtonian relaxation) and deep neural networks for this task, which are successfully tested for uncertainty quantification of stochastically forced oscillators and transmission failures in electrical power grids.

Business taxation, by affecting the costs of certain behaviors of firms, owners, or their counterparties, can trigger potentially substantial changes in real activity, such as changes in inputs or production processes. But it can also prompt avoidance responses---such as legal restructuring or changes in tax reporting---that may have important effects on efficiency and distribution. Understanding such responses is thus critical for enacting efficient and well-informed tax policy.

In this dissertation I investigate the real and avoidance responses at the intersection of several important topics in businesses taxation, namely capital taxation, taxation of passthrough entities, international taxation, and corporate taxation. My research sheds new light on our understanding of US business taxation by employing a variety of empirical methods to (1) develop new explanations for persistent puzzles in the literature, (2) fill knowledge gaps in the current body of business tax research, and (3) draw attention to new issues that have so far received little attention by public finance economists.

In Chapter 1, I investigate financing and investment responses by corporations to a change in capital taxation, presenting results that help resolve an existing conflict among empirical findings in the public finance literature. I estimate that dividend taxes, by impacting the cost of equity financing, have large effects on the financing, investment, and real outcomes of many US public firms. But---in contrast with economists' longstanding focus on capital investment outcomes---I find these responses are mostly from smaller, cash-constrained firms through “non-capital” investment channels: R&D and operating expenditures. Exploiting a quasi-experiment that tracks financing and expenditure responses to the 2003 dividend tax cut, I estimate a large, immediate, and sustained increase in average equity financing (+86% ± 11%) by these firms, reflecting a high elasticity to the cost of capital. Responsive firms put the cash substantially toward operating expenditures and R&D, rather than tangible investment. I also find higher job growth and long-run sales among the responsive firms. These results make sense, reconciling mixed evidence in recent research: because dividend taxes affect the cost of equity financing, the firms impacted most are those that actually rely on equity financing---smaller, often unprofitable, less capital-intensive firms who invest heavily in "non-capital" pathways.

In Chapter 2, I describe and estimate tax avoidance behavior that uses complex entity structures involving partnerships and tax havens to exploit discrepancies in tax treatment of capital income across jurisdictions. I also address a significant missing piece of knowledge in the public finance literature: where partnership income goes. Partnerships are the fastest growing class of business entity in the United States and represent over one third of reported business income, but due to their legal complexity, data quality, and opaque nature economists have not yet been able to identify where a sizeable portion of this income goes. In this paper, I use US federal tax records from 2005-2019 to compile a comprehensive analysis covering 99% of the income flowing to the owners of partnerships. I find that a much larger portion goes to foreign owners than previously thought, and that most of this amount goes to tax havens---over $1 trillion since 2011. The majority of these flows likely face zero tax in either the US or in the tax haven. The evidence I present suggests a prevalent use of entity arrangements by investment firms that shield investors from tax and reporting through "blocker structures," predominantly in the Cayman Islands. Evidence also suggests a substantial increase in income reported after the enactment of Foreign Account Tax Compliance Act (FATCA).

In Chapter 3, I investigate the degree to which corporations can manipulate their accounting of expenses to avoid taxes, and what effects this has on the corporate tax base. The investigation exploits a unique corporate tax reform in Texas that replaced a 4.5% profits tax with a broader 1% gross revenue tax and that eliminated almost all deductions, but still permitted corporations to deduct one of two categories of expenses: cost of goods sold (COGS) or total worker compensation. Data from federal corporate income tax returns makes it possible to estimate the effects of the reform, as data are consistent across years and harmonized across states. Strong evidence reveals a very large avoidance response for COGS but not for compensation: corporations reduced the tax base roughly 4% by reclassifying non-deductible expenses into COGS (with a large elasticity of roughly -5 ± 1), but there is little reclassification into compensation. These findings reveal the potentially very large but also highly context-specific nature of accounting reclassification responses. Given that numerous states have some form of gross receipts tax and that there is currently wide discussion of measures to broaden corporate tax bases by incorporating accounting measures, these findings offer important considerations for policymakers and tax authorities when designing, scoring, and enforcing corporate tax changes.

Disruptive technological innovation is the contemporary face of innovation and a dominant force in society. Change is occurring faster and upsetting existing scientific and technical policy systems. Entrepreneurs and innovators, drawing on a folk economic model of regulation, often believe that regulation cannot keep up with the pace of change and therefore policy makers should stay out of their way. Like many folk models, this perception of regulation-as-intrinsic-impediment-to-innovation may sometimes be true but it is not always true. Worse yet, this folk perception of regulators-as-impediment leads entrepreneurs and innovators to ignore opportunities to co-create beneficial regulations and instead create their own bad outcomes by prompting regulators to craft draconian regulations in response to entrepreneurs’ malicious non-compliance.Innovators thus oppose regulation not because they’ve had bad experiences but because they think they will in the future. A popular version of this folk economic model of regulation brandishes the word “disrupt” while storming the halls of stodgy industries and regulatory agencies. Despite this contemporary disruptive innovation narrative, substantial technological change is not a recent invention (though it may be accelerating). The reified economic rhetoric of the folk economic model has convinced disruptive entrepreneurs that regulation is a dirty word synonymous with state inadequacy. Although never perfect and sometimes inadequate, regulators have invariably adapted to technological change. This project explains how regulators have before, are now, and can again become allies of innovators when entrepreneurs look past limiting preconceptions. Regulatory scholars who study actually-existing regulation will recognize the folk economic model as an extreme version of “capture” within “command and control” regulation (c.f. Carpenter and Moss 2014b; Slayton and Clark-Ginsberg 2018). They have repeatedly demonstrated the deceptive inadequacy of totalizing catch-all models of regulation. Nevertheless, scholars who do not study actually-existing regulation often use this folk economic capture baseline to judge all work on regulation which hinders scholarly understanding of relationships between regulation and innovation (c.f. Dal Bó 2006; Carrigan and Coglianese 2015). With these scholarly limitations, lay entrepreneurs’ misperceptions are no surprise. Contrary to the folk model, I argue regulators have been, are now, and can again be so much more than merely a deadweight loss to innovation if only innovators and entrepreneurs can be guided past self-limiting imaginaries such as the folk economic model of disruptive innovation. To develop this argument, I derive a deductive typology of regulatory imaginaries and discuss how we can use this typology to understand the variety of relationships between regulators, entrepreneurs, and innovators that can lead to better or worse effects on innovation. I then specify my novel methodological approach of Bayesian Type Validation (BayesTV) which combines deductive typological theory with logical Bayesian analysis. Finally, I employ BayesTV to inductively verify my typology using three technological cases in the United States and European Union: autonomous vehicles (AVs), gene editing (GE), and electronic health records (EHR). The Folk Economic Model imaginary is but one of seven possible regulatory imaginaries of the proper relationship between regulators, entrepreneurs, and innovators. Regulatory imaginaries, based on the concept of sociotechnical imaginaries (Jasanoff 2015a), are collectively held, publicly performed conceptions of desirable relationships between regulation and technological innovation which actors believe are (or should be) institutionalized within regulatory agencies. Where the Folk Economic Model imaginary sees regulation as only an impediment to be minimized, the other six imaginaries see other potential effects such as moderation, constraint, and catalyst. Critically, my deductively derived and empirically validated typology also demonstrates that regulatory imaginaries are plural, diverse, and malleable. In presenting three empirical chapters covering multiple imaginaries, I demonstrate that there are plural actually-existing imaginaries around well know technologies. In presenting both similarities and differences in the US and EU implementations of regulation for each disruptive technology, I demonstrate that there is meaningful diversity among regulatory imaginaries in conceptual derivation, expected effect on innovation, and empirical implementation. Finally, in the application of BayesTV to the empirical cases, I demonstrate that regulatory imaginaries are malleable through policy. This project focuses on regulatory imaginaries because they shape the perceptions of what is possible and desirable about the relationship between regulators, entrepreneurs, and innovators around disruptive innovation. While future studies should build on this focus on imaginaries by exploring their origins and how contending imaginaries shape the outcomes of the policies that are built around them, this project focuses on the imaginaries themselves in order to demonstrate that we need not limit ourselves to the Folk Economic Model which sees regulation, as a rule, as merely deadweight.

Atomically thin van der Waals (vdW) materials are sensitive to their surrounding environment, and the properties they exhibit can be significantly affected by changing this environment. By placing two atomically thin layers together with different work functions, charge will transfer between the two materials. Appreciable hole doping (on the order of 1013 holes/cm2) is realized in graphene, MoS2, and WSe2 interfaced with α-RuCl3 as a result of this charge transfer. α-RuCl3 has a relatively large work function that allows it to accept a large density of electrons. The charge transfer to α-RuCl3 can be modified by increasing the distance between the vdW material and α-RuCl3 with a hBN spacer layer.The electrochemical intercalation of lithium ions in graphene and TMD heterostructures is shown to change significantly in the presence of α-RuCl3. Raman spectroscopy and electrical transport measurements are used to monitor the intercalation reactions in situ. The main finding is that α-RuCl3 significantly lowers the driving force needed to electrochemically insert lithium ions into vdW heterostructures. It is hypothesized that the hole doping induced by α-RuCl3 increases the favorability of electron reduction in the other layer that is concomitant with lithium intercalation. In addition, the intercalation driving force and the intercalation amount can be tuned with the doping in these heterostructures using hBN spacers. The control α-RuCl3 offers for ionic charge accumulation could potentially inform future battery technologies. The control over electronic charge accumulation in α-RuCl3 heterostructures also has interesting applications. The transport of graphene interfaced with monolayer α-RuCl3 is shown to exhibit an anomalous, dynamic response to gate voltages when observed by low temperature magnetotransport measurements. While the origin of this behavior is unknown, the characteristics of the transport in the presented graphene/α-RuCl3 devices could be useful for next generation electronic devices. When TMDs are interfaced with α-RuCl3, their trans- port properties are also significantly impacted. WSe2 doped with α-RuCl3 is metallic and its pristine photoluminescence (PL) response is quenched, which could have significant implications if utilized in optoelectronic devices. In addition, lateral heterojunctions of pristine WSe2 and WSe2 on α-RuCl3 exhibit pP type diode junction behavior, with direction dependent current responses. This study demonstrates the versatility of α-RuCl3 heterostructures, and documents the progress we have made towards honing our ability to control electronic and ionic charge in vdW heterostructures.

Polyketide synthases (PKSs) are versatile multimodular enzymes that biosynthesize molecules in an assembly line fashion. The discrete and sequential processing of the polyketide intermediates by domains in PKSs give rise to our ability to retrobiosynthetically predict the structure of the polyketide produced by a PKS simply by analyzing the amino acid sequence of said PKS. Likewise, much work has been done to rationally engineer novel PKSs to make designer polyketides by using retrobiosynthetic logic to stitch together PKS domains in the order required to produce a desired molecule. In this work, we sought to understand the engineering principles required to construct functional chimeras of PKSs with various reductive loop swaps to produce polyketides with varying degrees of reduction in Streptomyces albus. Furthermore, we sought to make the advantageous heterologous host, Pseudomonas putida, well adapted to produce polyketides, particularly free fatty acid-derived polyketides, by eliminating free fatty acid catabolic pathways and demonstrating the production of medium-chain free fatty acids. Finally, we present ongoing efforts to produce free fatty acid-derived polyketides that can give rise to valuable flavorant delta-lactones.