Searches for nonminimal Higgs bosons from a virtual Z decaying into a muon pair at the SLAC e+e storage ring

Nonminimal neutral Higgs bosons decaying into a muon pair were searched for using the Mark II detector at the SLAC e+e collider PEP at &s =29 GeV. A neutral scalar Higgs boson H, can be produced accompanied by a pseudoscalar neutral Higgs boson H via a virtual Z . If the mass of one of the Higgs bosons is between the muon pair threshold and the kaon pair threshold, it may decay predominantly into a muon pair. We looked for muon pair+jet(s) [e+e ~H, H~~p+p, +qq(r+r )] and three-muon-pair [e+e ~H, H~~3(H~~)~3(p+p )] topolo-gies. We found no evidence for these signals above the known background level, and we obtained limits on I (Z ~H, H~) as a function of the Higgs-boson masses.

Harvard University, Cambridge, Massachusetts 02138 (Received 17 February 1989) Nonminimal neutral Higgs bosons decaying into a muon pair were searched for using the Mark II detector at the SLAC e+e collider PEP at &s =29 GeV. A neutral scalar Higgs boson H, can be produced accompanied by a pseudoscalar neutral Higgs boson H via a virtual Z . If the mass of one of the Higgs bosons is between the muon pair threshold and the kaon pair threshold, it may decay predominantly into a muon pair. We looked for muon pair+jet(s) [e+ẽ H, H~~p+p, +qq(r+r )] and three-muon-pair [e+e~H, H~~3(H~~)~3(p+p )] topologies. We found no evidence for these signals above the known background level, and we obtained limits on I (Z~H, H~) as a function of the Higgs-boson masses.

I. INTRODUCTION: TWO-HIGGS-DOUBLET MODELS
We have searched for the production of light Higgs bosons decaying into muon pairs for a Higgs sector consisting of more than one doublet.
In the standard model, the Higgs sector is necessary to give mass to the weak gauge bosons as well as to the quarks and leptons. The Higgs boson is also necessary in order to prevent the cross section from violating (H, , H2, H3 ) and two charged Higgs bosons (H+ and H ). In the case of the neutral nonminimal Higgs bosons, H3 is a pseudoscalar and the other two are scalars, if their parity is defined through their couplings with fermions. To be more precise, H3 is a CP-odd state and the other neutrals (H& and H2) are CP-even states, if CP is conserved at the tree level. ' In this paper H, denotes a scalar Higgs boson and Hp denotes a pseudoscalar. We also use the notation H; and Hj for the two  Limits on some nonminimal Higgs-boson masses and couplings already exist. Some parameter ranges are excluded by searches for the standard-model Higgs boson.
Recently, CLEO has searched for 8-meson decay into a E meson plus a neutral Higgs boson, for the Higgs-boson decay modes H~p+p,~+~, EK, KE*, E*K, and K*K* (Ref. 12). They exclude the standard Higgs boson from mass ranges between 0.3 and 3.0 GeV and between 3.2 and 3.6 GeV. Chivukula and Manohar' summarized the light standard-model Higgs-boson searches in K and 8 decay and concluded that the standard-model Higgs boson has been excluded up to a mass of 2M, if the number of generations is three, and up to 0.36 GeV independent of the number of generations. The CUSB II group has searched for Y~H y and excluded the standard-Higgs-boson mass fram the region between the muonpair threshold and 5.0 GeV (Ref. 14). These limits also constrain the nonminirnal-Higgs-boson mass. However, since the nonminirnal-Higgs-boson couplings to the b quark and other fermions are model dependent, their masses in these regions are not excluded in a modelindependent way.
Searches for nonminirnal Higgs bosons from virtual Z 's have been performed at SLAC and DESY e+e colliders PEP and PETRA. Glashow

III. APPARATUS
The data analyzed here were acquired by two different configurations of the Mark II detector at PEP. The center-of-mass energy for all of the data was 29 GeV. An integrated luminosity of 210 pb ' was taken with the first configuration of the detector, which has been described extensively elsewhere. ' The components of the detector most important to this analysis were the tracking chambers and the muon system. Charged-particle tracks were measured by a 16-layer cylindrical drift chamber and a high-resolution vertex drift chamber in a 2.3-kG axial magnetic field. This combination gave a vertexconstrained momentum resolution of (crz/p) =(0.025) +(0.011p) (p in GeV). The drift chamber did not have multiple-hit readout capability.
The muon system consisted of planes at the top and bottom and both sides of the detector, each containing four layers of iron absorber followed by proportional tubes. The muon system covered 45% of the solid angle, and any muons outside that solid angle were not identified.
The electromagnetic calorimeters and time-of-Aight (TOF) system were used in background elimination. The electromagnetic calorimeter system consisted of eight modules in an octagonal array outside the magnet coil. The modules consisted of 37 layers of 2-mm-thick lead planes and 3-mm-thick liquid-argon gaps. Details of the liquid-argon system are described elsewhere.
The time- The signature for these events is an isolated muon pair with small opening angle. The event selection was started from 5.85 X 10 events on the data-summary tapes (DST's). The data-reduction cuts made before writing to these tapes which affected this analysis were mainly the following two cuts.
(DST1) The visible energy (charged and photon) of an event was larger than 0.25&s (0.15&s for data taken with the upgraded detector).
(DST2) The visible charged energy of an event was larger than 0.125&s (0.075&s for data taken with the upgraded detector).
The integrated luminosity corresponding to the data we analyzed is 225 pb . The experimental selection criteria for these events are the following.
(1) The total charged multiplicity of the event is greater than or equal to four. (2) There is at least one oppositely charged pair (i, j ) of tracks with opening angle f; satisfying f, (38'. This cut is optimized so that the muon pairs whose invariant mass is smaller than 2M, can be detected efficiently.
Hence, Higgs bosons with mass between 2M+ and 2M may also decay into muon pairs with a reasonably large branching fraction and be detected.
(4) The momenta of the two charged particles are Ip;I» g «V, Ip, j».g «V.
(5) Except for the particles i and j, no charged particles (p )0.2 GeV) are within a 60 degree cone from the momentum sum (p;+p~) of the pair. The total photon energy within the 60 degree cone is less than 1.0 GeV.
(6) Out-of-time cosmic muons, which are reconstructed as two parallel tracks in the chamber, are rejected by using the TOF counter information. The event is rejected if both tracks do not have a hit in a TOF counter or if the timing is far from nominal.
After these topology cuts, 945 events survived. Since these cuts alone were not enough to isolate the signal, the following lepton identification cuts were added.
(7) Both tracks must be identified as good muons: all four layers of the muon tubes must have hits within 3o. of the track extrapolation from the central drift chamber, taking into account fit and multiple-scattering errors. In order to hit the muon system, the polar angle of the track must satisfy~cos8~&0.45.
(8) Finally, e+e p+p events due to pure QED processes are rejected by removing the following events.
(a) Events with charged multiplicity equal to four and with at least one identified electron. An identified electron was defined as a track in the liquid-argon fiducial volume which had E;"/p )0.8, where E;" is defined elsewhere.
The electron cut was loose in order to eliminate as many QED events as possible.
(b) Events with at least one identified electron and with no reconstructed charged pions, which are defined as charged tracks which are neither identified electrons nor identified muons.
The detection eKciency for the events is obtained by running the analysis program on data generated by a Monte Carlo calculation with a full detector simulation.
For opening angles greater than 10', the detection efficiency is typically 15 -20%%uo. It does not depend much on the lighter-Higgs-boson mass unless the mass is very close to the muon-pair threshold. The eSciency for resolving the two muon tracks decreases with decreasing opening angle. If one considers angles less than 10', for the old Mark II chamber, the efficiency decreases to 7%%uo at M 0=0.22 GeV; for the upgraded Mark II drift t chamber, the efficiency is 13%, significantly better than for the old drift chamber. After all the cuts (1) -(8) three events survive. Measured parameters for the three surviving events are listed in Table I, where m + is the squared invariant mass P P for the muon pair, and m "";& is the squared invariant mass of the particles recoiling against the muon pair:

B. Background estimation
The most obvious sources of background are the twophoton processes, where one of the virtual photons is converted into a muon pair and the other into a quark or r pair [see Fig. 1(e)]. The expected number of background events from these processes after the cuts is 0.92+0. 14, estimated using the Berends-Daverveldt-Kleiss Monte Carlo program and incorporating the hadronic fragmentation of the quark pair by the Lund scheme.
The number of background events due to multihadronic events containing two real muons is estimated by the Monte Carlo calculation. If a B meson decays into pvD and the D into a pv plus a KL while the B meson decays into hadrons, for example, the event has an isolated pair of oppositely charged muons and no charged hadrons within that hemisphere produced from fragmentation; it would pass the cuts. Monte Carlo bb events were passed through the same analysis chain as the real data. The number of background events thus estimated for this category is 0.72+0.28.
We estimate the background due to fake muons, namely hadron punch-through or K+or m+decays into rnuons, by using the data and fake muon probabilities of hadrons. The expected number of background events for the case where only one of the isolated muon-pair tracks is a misidentified hadron is estimated from events satisfying all selection criteria except that only one of the pair tracks satisfied muon identification criteria while the other track is identified as a hadron. The number of background is calculated to be sum of the misidentification probabilities Pz "(p; ) for the hadron track in each of the nine such events found in the data. The probability Pz "(p; ) is given elsewhere. The sum is performed over the events surviving all the cuts except the muon identification [cut (7)]; instead, both of the pair tracks are categorized as hadrons (each of the two identified neither as an electron nor as a muon). The estimated background is 0.015+0.004. The error is dominated by the systematic error in the misidentification probability.
Therefore, the expected number of background events due to misidentified muons is small (about 0.10 events). The summary of the background is listed in Table II. The total number of background events is estimated to be 1.73+0.32, while the number of observed events is three. Therefore, all of the three events are consistent with background. I.
dt's P P where o is the v"v"cross section at v's =29 GeV, e is P P the detection efficiency, Po is the P value on the Z peak. This method eliminates the P threshold effect for heavy Higgs bosons at &s =29 GeV so that the limit can be compared with future results at the Z peak from SLC and the CERN collider LEP. For the most optimistic case, shown by the "maximum-width" curve (no reduction of the cross section by the mixing) in Fig. 3(a), the heavier-Higgs-boson mass can be excluded up to about 12 GeV with 90%%uo C.L. The 90%-C.L. upper limit of the number of signal events s&; is obtained by using a likelihood method, taking into account the number of expected background events and its estimated error, the background distribution m"";t vs m"", and the errors in the invariant mass squared (m""andm"";i) in the observed three events.
The details of the likelihood function are described in the Appendix.
In order to have a somewhat model-independent result, the limit is obtained for the partial width of the Z decay into H; +H. normalized to the Z~v"v"width multiplied by the Higgs-boson branching fraction into a muon pair. Figure 3 The signature of the events we are looking for is three isolated muon pairs with a small opening angle for each pair. The event selection began with the data-summary tapes (DST's) as in Case I. We applied a set of simpler experimental cuts for this case: (1A) The total charged multiplicity of the event is between five and seven; (2A) the total visible charged particle energy of the event exceeds &s /2; (3A) the event has three or more good muons: all four layers of the muon tubes have hits wi. thin We also tried another set of cuts which do not depend on the muon detector system: (18) The total charged multiplicity of the event is equal to six, with all six charged particles having p )0.7 GeV; (28) the visible charged-particle energy (assuming the pion mass for each particle) must exceed 26 GeV; (38) the total missing momentum normalized to the scalar sum of the six rnomenta must be less than 0.3; (48) there exists a combination of the six prongs such that three oppositely charged particle pairs have invariant masses which coincide to within 0.20 GeV when each pair is allowed to be e+e p+p, m m, or %+K . This allows us to find combinations of e+e +p+p +p+p, etc. The opening an-   Fig. 3(a), Fig. 3(b) shows the limit of the partial width of the Z decay into H, +H, normalized to the Z~v"v"width, as a function of the heavier Higgs-boson mass. Figure 4(

ACKNOWLEDGMENTS
We would like to thank Howard Haber for many useful discussions. This work was supported in part by Depart- The second function PBk (b) is the number of background events b assuming a Gaussian distribution: 20b exp where b0 is the estimated number of background events (b0=1.73) and o& is its error (ob=0. 32) as listed in Table II. This function restricts the number of background events b to be close to the estimated value b0.
The factor Q"(sf"+bg")I(s+b) is the likelihood, given n observed events, that they will have particular measured values of m + and m """i. The functions f and g are the probability distribution of signal and background events in m""vs m "";&. is a sum of events due to processes which contain a p+ p pair (such as e+e -+p+p qq) and events due to fake muons. Monte Carlogenerated events are used to evaluate the former background distribution, and real events with isolated p*h or h+h pairs are used for the latter case. The function g" is a good description of the background distribution, which has limited statistics. Varying a and A, has little effect on the resulting limits.
The combined factor (sf"+bg")/(s+b)is then the probability distribution for signal plus background. This probability is evaluated for the three events, and the results are multiplied together to get the likelihood.
The 90go-C.I.. limit of the number of signal events s&; is defined by f ' ds f db X(s, b)