An experimental bond-trading program being run at the Federal Reserve Bank of New York could fundamentally change the way the central bank sets interest rates.

Fed officials see the program, known as a "reverse repo" facility, as a potentially critical tool when they want to raise short-term rates in the future to fend off broader threats to the economy. Of particular concern for the Fed is finding a way to contain inflation once the trillions of dollars it has sent into the financial system get put to use as loans.

Like the plumbing beneath the floorboards in a home, this program is unseen by most investors and it isn't part of the Fed's current efforts to spur growth. Launched this year, it is still in a testing stage and isn't expected to be fully implemented for years.

Under this system, the Fed would raise short-term interest rates by borrowing in the future against its large and growing securities portfolio.

The Fed traditionally has managed short-term interest rates by shifting its benchmark federal-funds rate, an overnight intrabank rate. It did this by controlling how much money flowed into and out of the banking system on a daily basis. Small adjustments could significantly impact short-term rates. Since the Fed has pumped $2.5 trillion into the economy by purchasing bonds, the old system won't work unless the central bank pulls much of this money out. Instead, what Fed officials increasingly envision is a system in which it would tie up this money as needed by offering investors and banks interest on their funds.

The stakes are enormous. Right now banks aren't lending this money aggressively. But as the economy expands and lending picks up, the Fed will need to tie up the money to ensure it doesn't cause the economy or financial markets to overheat.

"The Federal Reserve has never tightened monetary policy, or even tried to maintain short-term interest rates significantly above zero, with such abundant amounts of liquidity in the financial system," according to a draft of a new research paper by Brian Sack, the former head of the New York Fed's markets group, and Joseph Gagnon, an economist at the Peterson Institute for International Economics and a former Fed economist.

Short-term rates—which serve as a benchmark for long-term rates for mortgages, car loans and other borrowing—are now near zero and the Fed doesn't plan to raise them for a couple of years. In normal times, the Fed cuts short-term rates to spur growth and raises them when it wants to slow growth.

When it does want to raise rates, the Fed under the repo program would use securities it accumulated through its bond-buying programs as collateral for loans from money-market mutual funds, banks, securities dealers, government-sponsored enterprises and others.

The rates it sets on these loans, in theory, could become a new benchmark for global credit markets.

The head of the New York Fed's markets group, Simon Potter, in a speech at New York University this month, described the new program as "a promising new technical advance."

Some market observers are going a step further and arguing that the Fed should abandon the fed-funds rate as its main lever for managing a broad spectrum of rates in the financial system.

Mr. Sack is among them. In his draft paper with Mr. Gagnon, they say the reverse-repo program should become a linchpin for the way the Fed manages interest rates in the future.

Mr. Sack, who is now co-director of global economics at the hedge fund D.E. Shaw, and Mr. Gagnon argue the Fed should discard its effort to target the fed-funds rate and instead use these repo trades as a primary way of guiding the borrowing rates that ripple through the economy.

The paper is notable because Mr. Sack, during his tenure at the New York Fed from 2009 to 2012, led the central bank's effort to find new ways to manage short-term interest rates.

Barclays analyst Joseph Abate said the repo program appears to have set a floor under short-term lending rates even during the small-scale tests. The general-collateral rate—the borrowing rate for the most common type of repo—has recently settled at about 0.10%, about 0.05% above the fixed rate the Fed has set for the repo facility and about 0.05% higher than it was earlier in the fall before the tests were launched.

Mr. Abate also said the Fed should abandon its fed funds target and stick with this program.

"The Fed has a very powerful tool on its hands," he said.

Without new tools like the repo facility, the Fed might not be able to control interest rates or it might be forced to take financially destabilizing steps such as selling the securities in its portfolio in a hurry.

The idea for the repo program bubbled up from the New York Fed's market group in part because another tool wasn't working well. Officials had seen a program known inside the Fed as IOER, for "interest on excess reserves," as the main avenue for managing short-term rates amid the flood of money in the system. Under this program, the Fed pays banks 0.25% for cash they keep at the central bank. In theory, when the Fed wants to raise short-term rates, it would raise this interest rate. Rather than lend out money, banks should want to keep it with the Fed.

In reality, the IOER program hasn't worked well, in part because some big market players including Fannie Mae and Freddie Mac can't participate. The fed funds rate has hovered well below the Fed's 0.25% floor for years. That isn't a problem now because the Fed wants to hold rates very low, but it raises concerns that the central bank won't have tight control of rates when the time comes to raise them.

The reverse-repo program extends the Fed's reach beyond traditional banks to Fannie, Freddie and others, and in theory should give the central bank more control over interest rates. Mr. Sack and Mr. Gagnon say the Fed should use the IOER program in conjunction with the reverse-repo program to set rates.

The Fed has been testing the repo program with 139 different counterparties in the past few months, including 94 money-market funds, and setting an interest rate of 0.05%.

"By reaching financial institutions that are ineligible to earn [interest on excess reserves]…the facility widens the universe of counterparties that should generally be unwilling to lend at rates below those rates available through the central bank," Mr. Potter said in his speech.