Health Secretary Jeremy Hunt said changes to the way GPs provide out-of-hours care have had a "huge impact" on accident and emergency services as a group of NHS trusts warned casualty departments could collapse in six months' time as a result of rising demand.
The Cabinet minister admitted there were "huge pressures" on accident and emergency services, including a rising number of frail elderly patients with dementia.
He said these pressures included a "dramatic fall in confidence" in alternatives to accident and emergency since the last government changed the GP contract in 2004 to remove responsibility for out-of-hours care from GPs.
"There are pressures throughout the economy and on every department and we have protected the budget for the health service but that doesn't mean there aren't huge pressures," he told ITV's Daybreak.
"What we need to do is to have a very fundamental look at the way A&E departments work and in particular look at the alternatives to A&E because the government changed the GP contract in 2004 and they removed responsibility for out-of-hours care from GPs. That has caused a dramatic fall in confidence in the public in what their alternatives to A&E are - that is what we have to sort out."
In an interview with BBC Breakfast, Mr Hunt said since the contract was changed there were now four million more people visiting A&E departments every year. "We need to look at the system. We need to understand that this is partly about how effectively we deal with people inside an A&E department but it is also about making sure that people have good alternatives to A&E," he said.
"I think one of the problems we have at the moment is that it is too difficult to access out-of-hours care. People don't feel confidence in the care they will get, if they speak to a GP, the GP probably won't be able to see their medical notes and know about their background."
Pressed on BBC Breakfast whether he was going to change the GP contract, Mr Hunt said: "That is one of the things we need to look at, but I want to stress that it is not the only thing. There are other things like joining up the health and social care system, making sure that people get better cared for in residential care homes."
Mr Hunt's comments come after the College of Emergency Medicine called for "fundamental change" in the way emergency care is run, warning that A&E units are facing their biggest challenge in more than a decade as departments grapple with "unsustainable workloads" and lack of staff.
The Foundation Trust Network, which represents more than 200 health trusts in England, warned that A&E services were in danger of collapse in six months' time as a result of "huge pressure".