Opinion by Judge BETTY B. FLETCHER; Dissent by Judge BRUNETTI.

Frank Regula appeals the district court’s denial of his motion for summary judgment on his claim that the Delta Family-Care Disability Survivorship Plan (“Delta Plan” or “Plan”) abused its discretion in terminating his disability benefits. Specifically, Regula contends that the Delta Plan should have accorded deference to the opinions of his treating physicians and considered vocational evidence in making its benefits determination. Regula also sought summary judgment on the ground that the Delta Plan failed to provide him with a full and fair review of his disability claim. The district court denied Regula’s summary judgment motion in its entirety and entered judgment in the Plan’s favor pursuant to a stipulation signed by the parties. We vacate the judgment of the district court and remand for a determination as to whether the Delta Plan may have been acting under a conflict of interest, and thus whether the court should have applied a less deferential standard of review to the Plan’s decision to discontinue Regula’s benefits.

I.

The Delta Plan is a non-contributory employee welfare benefit plan governed by the Employee Retirement Income Security Act of 1974 (“ERISA”), as amended. The Plan provides short and long-term disability benefits to non-pilot Delta employees.

Under the Plan, a participating employee is eligible for short-term disability benefits “when he is disabled as a result of a demonstrable injury or disease (including mental or nervous disorders) ... which prevents the Employee from engaging in his customary occupation.” An eligible employee can qualify for short-term disability benefits for up to eighteen months with the approval of the Plan’s Administrative Committee. Once the short-term disability benefits expire, a participating employee is eligible for long-term disability benefits if “he is disabled at that time as a result of demonstrable injury or disease (including mental or nervous disorders) which will continuously and totally prevent him from engaging in any occupation whatsoever for compensation or profit, including part-time work.” An employee is eligible for long-term benefits “so long as he remains disabled.” In addition, an employee must be “under the care of a physician or surgeon for the injury or disease” to remain eligible for such benefits. The Administrative Committee determines whether a participant is “disabled” and is therefore eligible for benefits.

The Administrative Committee also serves as the “named fiduciary” of the Plan and has “authority to control and manage the operation and administration of the plan.” As part of its function, the Administrative Committee is given “the exclusive power to interpret [the Plan]” and “its interpretation and decisions [are] final and conclusive.” Furthermore, the Committee is charged with “deciding] all questions concerning the Plan.”

The Plan has a two-tiered review procedure governing appeals from a claims denial. The Plan’s Administrative Subcommittee provides the first level of review. If the claimant is not satisfied with the outcome of the Subcommittee’s decision, the claimant may then appeal to the full Administrative Committee. During either level of the review process, the claimant or his representative is entitled to review *1134 “pertinent documents relating to the denial and may submit written comments.”

Frank Regula began his employment with Western Airlines (“Western”) in 1971 arid functioned normally in his job until sustaining “a severe injury to the neck, right shoulder and arm during the course of his employment” as a clerk in October 1985. Nearly two years later, Regula filed a claim for short-term disability benefits, claiming that he had sustained a “cervical disc injury.” The Delta Plan granted Re-gula’s request and awarded him short-term disability benefits through October 30,1987.

The Plan then awarded Regula long-term disability benefits beginning in November 1987 and approved the continuation of these benefits on thirteen separate occasions. The Plan reviewed Regula’s disability award every three months, and notified him each time that he would be required to submit an updated physician’s report to prove his continued eligibility for benefits. Throughout this period, Regula continued to demonstrate his eligibility by submitting updated reports from his treating physicians. On July 25,1995, however, the Plan terminated Regula’s long-term disability benefits because it determined that he was capable of working, rendering him ineligible for benefits under the definition of “Long Term Disability” in section 4.03 of the Plan.

Regula submitted two contemporaneous reports by his treating physicians in support of his claim for continued disability benefits. 1 In the first report, Dr. Sandra Smith, a psychologist, opined that it was “very probable” that Regula was still disabled in the summer of 1994. In response to an inquiry from the Plan about the possibility of vocational rehabilitation, Dr. Smith further clarified her diagnosis by declaring that Regula’s “combined physical and emotional symptom complex” prevented him from enrolling in a vocational rehabilitation program. As a psychologist, Dr. Smith stated that her diagnosis was limited to Regula’s psychological condition and that she would “defer ... to an appropriate medical specialist concerning Regula’s physical symptoms.”

Regula’s second report, prepared by Dr. Dean H. Cummings, provided a diagnosis of Regula’s continuing physical symptoms. In that report, Dr. Cummings concluded that Regula was still “permanently disabled” in December 1994 due to the “undesirable effects of multiple surgeries.” Specifically, Dr. Cummings cited Regula’s “DQnability to sustain repeated or prolonged standing, sitting, pulling, pushing, bending, stooping, lifting to waist level, lifting over [his] head, walking, exerting or sitting for 10 minutes or more” as disabling factors.

In response to Regula’s two reports, the Delta Plan arranged for Regula to be examined by Dr. Rajeswari Kumar, a specialist in physical and rehabilitative medicine. Dr. Kumar diagnosed Regula with chronic pain syndrome, post-traumatic pain in the cervical and lumbar regions, status post-anterior diskectomy at C5-6 and C6-7, and status post-decompression of superficial radial nerve, right upper extremity. Nevertheless, Dr. Kumar concluded that Regula was “definitely capable of gainful employment performing some *1135 type of work.” Dr. Kumar also noted that Regula could perform several specific activities including, most notably, standing and walking for four to six hours per day. These considerations led Dr. Kumar to conclude that Regula was not disabled.

In addition, the Plan arranged for a psychiatrist, Dr. James O’Brien, to assess Regula’s psychological condition. After examining him, Dr. O’Brien concluded that Regula was “consciously exaggerating his psychological and orthopedic difficulties.” In response to the report prepared by Regula’s psychologist, Dr. Smith, Dr. O’Brien specifically attacked Dr. Smith’s diagnosis as biased because she expressed opinions about Regula’s physical condition that were outside of her field of expertise. Consistent with Dr. Kumar’s report, Dr. O’Brien also concluded that “Mr. Regula can return to work immediately and that there is no type of work within his job description that he would not be able to do.” Thus, both of the Delta Plan’s examiners concluded that Regula was not disabled.

Based on these reports, the Plan terminated Regula’s long-term disability benefits in a letter dated July 25, 1995. The letter also notified Regula that the Plan would reconsider its decision, but that he must submit “objective evidence that [he is] continuously and totally disabled from engaging in any occupation or work for compensation or profit” in order to perfect his claim.

Regula then appealed the denial of his long-term disability benefits to the Administrative Subcommittee. The Plan sent Regula’s attorney, Lawrence Rohlfing, a letter informing him about a claimant’s right “to review pertinent documents related to [his] appeal and submit other comments, issues or evidence in further support of the appeal to the Subcommittee.” In response, Rohlfing submitted additional superannuated reports from Drs. Shapero and Smith, along with a four-page letter setting forth Regula’s legal and factual case for continued eligibility. Nevertheless, the Administrative Subcommittee rejected Regula’s appeal. The Plan communicated its rejection through a letter citing specific parts of the Plan’s examining physicians’ reports that supported the Subcommittee’s decision.

After the rejection of his appeal on the first level of review, Regula appealed to the full Administrative Committee. The Secretary of the Administrative Committee wrote a letter to Rohlfing delineating the types of evidence that could be submitted to perfect Regula’s claim. This letter was received only six days prior to the scheduled date of the hearing. Recognizing that the letter did not grant Regula a sufficient amount of time to collect and submit additional evidence, the Plan offered him an extension of time. Regula refused the Plan’s offer. Following a hearing, the Administrative Committee notified Regula that it was rejecting his appeal and informed him that he was no longer eligible for long-term disability benefits under the terms of the plan.

Regula subsequently filed an action against the Delta Plan in federal district court on October 17,1996, alleging that the Plan’s denial of his long-term disability benefits violated ERISA. On February 17, 1998, the district court denied Regula’s motion for summary judgment. The district court held that substantial evidence supported the Plan’s decision and that the Plan did not abuse its discretion as a matter of law. In finding for the Plan, the district court rejected several arguments that it viewed as a circumvention of the broad discretion afforded to Plan Administrators. First, the district court refused to apply the “treating physician rule” in an ERISA case. Second, the district court *1136 rejected Regula’s contention that the Plan’s failure to consider vocational evidence was an abuse of discretion. Finally, the district court found that the Plan had conducted a full and fair review of Regu-la’s claim.

Following the denial of Regula’s summary judgment motion, the parties agreed to the following stipulation:

The district court approved the stipulation by entering judgment. Regula then filed this appeal pursuant to 28 U.S.C. § 1291 , which grants this court appellate jurisdiction over a final decision of the district court.

II.

A district court’s denial of summary judgment is ordinarily not appeal-able. California v. Campbell, 138 F.3d 784 , 786 (9th Cir.1998); see also Behrens v. Pelletier, 516 U.S. 299, 306-07 , 116 S.Ct. 834 , 133 L.Ed.2d 773 (1996) (holding that a denial of immunity is within a small class of cases that are immediately appealable from a denial of summary judgment). The so-called final judgment rule allows parties to appeal only final decisions of the district courts. See 28 U.S.C. § 1291 . A final judgment is “ ‘a decision by the District Court that ends the litigation on the merits and leaves nothing for the court to do but execute the judgment.’ ” Dannenberg v. Software Toolworks Inc., 16 F.3d 1073, 1074 (9th Cir.1994) (quoting Coopers & Lybrand v. Livesay, 437 U.S. 463, 467 , 98 S.Ct. 2454 , 57 L.Ed.2d 351 (1978)). The purpose of the final judgment rule is to ensure that all claims are raised in a single appeal to avoid piecemeal adjudication of a single controversy. See Cobbledick v. United States, 309 U.S. 323, 325 , 60 S.Ct. 540 , 84 L.Ed. 783 (1940).

As an appellate court, we are required to raise the finality of the district *1137 court’s order sua sponte on appeal. See In re Exennium, Inc., 715 F.2d 1401, 1402 (9th Cir.1983). As we noted in Dannen-berg, this court takes a “pragmatic approach to finality,” and there are situations when a stipulated judgment may fulfill the purposes underlying the final judgment rule. 16 F.3d at 1075 . These limited cases arise only when all the claims that may be the subject of an appeal have become sufficiently finalized. See id. Thus, although the parties in this case stipulated to the judgment, we have jurisdiction only if the stipulated judgment sufficiently finalized the district court’s order denying Regula’s summary judgment motion. We conclude that the stipulated judgment so finalized Regula’s appeal.

The district court’s denial of Regula’s motion for summary judgment conclusively decided the legal and factual issues in the case. As stated in the stipulation, the parties agreed that Regula could not prevail at trial after the denial of his summary judgment motion. Accordingly, the parties stipulated to the judgment to facilitate the immediate appeal of an ordinarily non-appealable order in an effort to “preserve scarce resources.” The stipulation placed all the issues before this court for de novo review, preserving resources because there were no undecided issues that could subject this court to the threat of piecemeal adjudication through multiple appeals. The district court then properly finalized the stipulation by entering judgment in favor of the Plan.

In a nearly identical case, we held that parties may stipulate to a final judgment if all the issues in the case are placed in this court on appeal. See Comsource Independent Foodservice Cos. v. Union Pacific Railroad Co., 102 F.3d 438 (9th Cir.1996). In Comsource, the defendants filed a summary judgment motion, claiming that the plaintiffs cause of action was barred by the statute of limitations. See id. Although the district court denied summary judgment, the parties stipulated to a final judgment for the purpose of facilitating an appeal. See id. As in this case, all the issues in Comsource were decided by the district court and placed in this court for de novo review. The stipulation turned a nonfinal order — the denial of a summary judgment motion' — into a final judgment by virtue of an agreement between the parties and subsequent approval by the district court through an entry of judgment. See id. The procedure set forth in Comsource directly controls the jurisdictional question in this case. 2

Thus, under the controlling authority of Comsource, we have jurisdiction over this *1138 matter pursuant to 28 U.S.C. § 1291 . The scope of our review is limited to the district court’s denial of summary judgment. See Comsource, 102 F.3d at 442 .

III.

We review a district court’s denial of summary judgment de novo. See Moran v. Washington, 147 F.3d 839, 844 (9th Cir.1998). On review of summary judgment, we must determine whether the evidence, when viewed in a light most favorable to the nonmoving party, raises any genuine issues of material fact and whether the district court correctly applied the substantive law. See Berry v. Valence Tech., Inc., 175 F.3d 699 , 703 (9th Cir.1999). We also review de novo the district court’s choice and application of the standard of review applicable to decisions of plan administrators in the ERISA context. Lang v. Long-Term Disability Plan of Sponsor Applied Remote Tech., Inc., 125 F.3d 794, 797 (9th Cir.1997).

Due to the unusual posture in which this case has come before us, we make special note to address the parties’ stipulation, which has been ratified by the district court. The parties have stipulated to the abuse of discretion standard following the district court’s ruling. This stipulation was undertaken by the parties with the apparent purpose of granting finality to the ruling of the district court on summary judgment. However, the decision of the parties to set aside issues of law does not affect the scope of our review.

As an appellate court, we are not bound by stipulations as to questions of law. Estate of Sanford v. Commissioner of Internal Revenue, 308 U.S. 39, 51 , 60 S.Ct. 51 , 84 L.Ed. 20 (1939); Swift & Co. v. Hocking Valley Ry. Co., 243 U.S. 281, 289-290 , 37 S.Ct. 287 , 61 L.Ed. 722 (1917) (stating that a stipulation intended to be “treated as an agreement concerning the legal effect of admitted facts ... is obviously inoperative; since the court cannot be controlled by agreement of counsel on a subsidiary question of law”). An appellate court “aet[s] without any impropriety in refusing to accept what in effect [is] a stipulation on a question of law.” United States Nat’l Bank of Oregon v. Indep. Ins. Agents of Am., Inc., 508 U.S. 439, 448 , 113 S.Ct. 2173 , 124 L.Ed.2d 402 (1993). Furthermore, the Supreme Court has stated that “a court of appeals does not abuse its discretion when it raises the validity of a law even when the parties failed to raise the issue in the briefs or before the district court.” United States v. Alameda Gateway Ltd., 213 F.3d 1161, 1167 (9th Cir.2000) (citing Nat’l Bank of Oregon, 508 U.S. at 448 , 113 S.Ct. 2173 ).

For reasons discussed below, we set aside the parties’ stipulation as to the appropriate standard of review and remand to the district court for further proceedings.

A.

Defining the proper standard of review to apply to the administrator’s benefits determination is, of course, critical to determining the outcome here. The parties’ stipulation is suggestive of how it might be critical, but again not dispositive on the issue of what the standard in fact should be or how it ought to be applied. Judge Brunetti hypothesizes in his dissent that, by stipulating to an abuse of discretion standard, the parties have implicitly stipulated that the administrator’s decisions were not impaired by a conflict of interest. Dissent at 13511. We reject this interpretation for several reasons. First, the record below is insufficiently developed, and the language of the stipulation insufficiently rich, for us to infer such a crucial point. Because the district court did not solicit *1139 arguments concerning the apparent existence of a conflict, the parties’ views on the subject are impossible to determine. There is no indication that the stipulation was entered in order to restrict our appellate review of the facts of this case beyond the restrictions ordinarily placed on the review of a ruling on summary judgment. 3

Second, as discussed in greater detail below, our precedent in the area of ERISA disability determinations refers to both a highly deferential and a less deferential standard of review by the same term, “abuse of discretion.” 4 Precisely because the degree of deference owed to the decision maker is split within the standard according to the factual conditions under which the standard is to be applied, stipulation to the standard cannot be said to imply agreement as to the crucial factual predicate upon which the degree of deference turns.

Third, and diametrical to the dissent’s position, we might infer from the parties’ choice to place the treating physician rule at the center of this controversy that, without recourse to such a rule, the appellant would be denied even a threshold opportunity to examine the sufficiency of the Plan’s reasons for terminating benefits, precisely because the Plan would be under no duty either to make their reasons specific or to base them on substantial evidence. We do not agree that this inference should guide our decision either, in part because we do not assume that the abuse of discretion standard under ERISA is necessarily incompatible with importation of the treating physician rule.

The treating physician rule applied in the Social Security setting requires that the administrative law judge (“ALJ”) determining the claimant’s eligibility for benefits give deference to the opinions of the claimant’s treating physician, because “he is employed to cure and has a greater opportunity to know and observe the patient as an individual.” Morgan v. Comm’r of Soc. Sec. Admin., 169 F.3d 595, 600 (9th Cir.1999); see also Smolen v. Chater, 80 F.3d 1273, 1285 (9th Cir.1996). This grant of deference to a treating physician’s opinions increases the accuracy of disability determinations, by forcing the ALJ who rejects those opinions to come forward with specific reasons for his decision, based on substantial evidence in the record. Just as in the Social Security context, the disputed issue in ERISA disability determinations concerns whether the facts of, the beneficiary’s case entitle him to benefits. Therefore, for reasons having to do with common sense as well as consistency in our review of disability determinations where benefits are protected by federal law, we see no reason why the treating physician rule should not be used under ERISA in order to test the reasonableness of the administrator’s positions. 5

*1140 The dissent rejects this comparison on the belief that “a disability determination under ERISA is subject almost entirely to the plan’s language.” Dissent at 13515. The dissent further contends that adaptation of the treating physician rule to ERISA would impermissibly erode the discretion granted to a plan administrator under the terms of a lawful benefits contract. Dissent at 13515. We disagree with both of these propositions.

First, we note that the deference given to a treating physician’s opinions under the rule is not absolute. “When a nontreating physician’s opinion contradicts that of the treating physician-but is not based on independent clinical findings, or rests on clinical findings also considered by the treating physician-the opinion of the treating physician may be rejected only if the ALJ gives ‘specific, legitimate reasons for doing so that are based on substantial evidence in the record.’ ” Morgan, 169 F.3d at 600 (quoting Andrews v. Shalala, 53 F.3d 1035, 1041 (9th Cir.1995)). “[I]f the treating physicianfs] opinions are uncontroverted, those reasons must be clear and convincing.” Smolen, 80 F.3d at 1285 . The opinions of a nonexamining (or reviewing) physician may serve as substantial evidence under the rule, when they are supported by other evidence in the record and consistent with the evidence in the record overall. See Shalala, 53 F.3d at 1041 . Therefore, the discretion of the plan administrator would be no more abrogated by compliance with the treating physician rule than it ought to be under a statute the purpose of which is at least in part “to promote the interests of employees and their beneficiaries.” Shaw v. Delta Air Lines, Inc., 463 U.S. 85, 90 , 103 S.Ct. 2890 , 77 L.Ed.2d 490 (1983).

Second, we reject the view that disability determinations under ERISA are determined almost exclusively by plan language because we believe that this misstates the true role of plan language in determining our standard of review. The Supreme Court has held that our review of benefits determinations by an insurer will be for abuse of discretion when the plan language designates discretion to make such determinations and to interpret eligibility with the plan administrator. Firestone Tire & Rubber Co. v. Bruch, 489 U.S. 101, 115 , 109 S.Ct. 948 , 103 L.Ed.2d 80 (1989). This ruling, however, does not indicate precisely how reviewing courts ought to assess the reasonableness of an administrator’s determinations, only that ruling upon compliance with ERISA requires such an assessment.

Furthermore, we do not believe that the treating physician rule is inconsistent with *1141 the broad discretion accorded plan administrators under Firestone. Indeed, it is inaccurate to say that disability determinations under ERISA are almost entirely subject to the plan’s language. 6 While this statement may be presumptively correct with respect to a plan administrator’s authority to interpret the terms of a plan, in the context of a disability claim such as Regula’s, the key issue in determining whether a claimant is entitled to benefits is not the language of the plan, but the facts of the particular case. Hence, just as in Social Security cases, the issue in dispute in this case is not the meaning of the terms of the plan but whether the facts of Regula’s case entitle him to benefits. As with Social Security cases, eligibility for benefits turns almost entirely on medical professionals’ opinions as to the claimant’s impairments and residual capacity (i.e., whether or not Regula’s condition prevents him from performing “any occupation”).

Like the plan administrator under ERISA, the ALJ is given broad discretion to determine eligibility for disability benefits under the Social Security Act (“SSA”). Courts review these determinations under an abuse of discretion standard, and the treating physician rule assists in this review by ensuring that the ALJ’s decisions are based upon substantial evidence. It has long been settled among the circuit courts that disability determinations under SSA will be guided by the treating physician rule. See Murray v. Heckler, 722 F.2d 499, 501 (9th Cir.1983) (joining the Second, Fifth, and Sixth Circuits in holding that the treating physician’s opinions should be accorded greater weight than that of an examining or reviewing physician). The Social Security Administration subsequently codified the treating physician rule in its regulations governing disability determinations. See 20 C.F.R. §§ 404.1527 (d), 416.927(d) (2001).

Under ERISA, the plan administrator is similarly charged with the task of making accurate disability determinations and those determinations are reviewable by courts in order to ensure that they are based upon appropriate and substantial evidence. See Eley v. Boeing Co., 945 F.2d 276 , 278 n. 1 (9th Cir.1991) (clarifying that the abuse of discretion standard under ERISA requires the plan administrator to support its decisions by “substantial evidence”). Review of the sufficiency of evidence supporting a plan administrator’s disability determinations, as well as the consistency of the administrator’s actions in dealing with the beneficiary, have long been a part of our review of disability determinations pursuant to the ERISA abuse of discretion standard. To the degree that the treating physician rule can assist courts to enforce the accuracy of disability determinations under ERISA, we find no reason why the rule should not be adapted to that context.

The dissént further complains that adaptation of the treating physician rule to the ERISA context is beyond our judicial authority because Congress has implicitly excepted disability determinations under *1142 ERISA from compliance with the rule. The dissent argues that, because the rule has been codified under the Social Security Act and yet Congress failed to codify it under ERISA, congressional inaction should be interpreted as a rejection of the rule’s propriety in the ERISA context. Dissent at 13515. This argument is based upon a false assessment of the genesis of both the standard of judicial review under ERISA and the treating physician rule under the Social Security Act.

In responding to the dissent, it is important to recognize that ERISA itself does not designate a particular standard for judicial review of plan administrators’ disability determinations. Instead that standard was designated by the Supreme Court in Firestone Tire & Rubber Co. v. Bruch, supra, according to its interpretation of the purposes of the statute. In Firestone, the Court stated that “ERISA was enacted ‘to promote the interests of employees and their beneficiaries’ and ‘to protect contractually defined benefits.’ ” 489 U.S. at 113 , 109 S.Ct. 948 (citation omitted) (quoting Shaw, 463 U.S. at 90 , 103 S.Ct. 2890 , and Mass. Mutual Life Ins. Co. v. Russell, 473 U.S. 134, 148 , 105 S.Ct. 3085 , 87 L.Ed.2d 96 (1985)). It is for the latter reason that the Court adopted the abuse of discretion standard where plan language ceded discretion'to the plan administrator to determine benefits and interpret contractual language. Id. at 115, 109 S.Ct. 948 . We address the standard established by the Firestone decision in greater detail below. For the moment, it is most important to note that the Court’s ruling was intended to reconcile the joint objectives of the statute (rather than to satisfy one to the exclusion of the other) while simultaneously interpreting the provisions of ERISA in order to maintain their consistency with “other settled principles” of law. Id. at 112 , 109 S.Ct. 948 .

Before fashioning this compromise, the Court first had to address the argument that to determine independently a standard of review was beyond its institutional authority. The petitioner Firestone had argued that, because Congress failed to pass a bill that would have overturned circuit court precedent applying an arbitrary and capricious standard to ERISA claims, such inaction indicated that Congress itself agreed with the application of that standard. Id. at 114 , 109 S.Ct. 948 . The Court disagreed, stating that “[tjhough ‘instructive,’ failure to act on the proposed bill is not conclusive of Congress’ views on the appropriate standard of review.” Id. In adjudicating between the arbitrary and capricious and de novo standards advocated by the parties, the Court concluded that the question of what standard to apply would turn partially upon plan language. It further modified this rule, in order to preserve its consistency with established principles of trust law, by stating that, even where plan language places discretion with the administrator, a conflict of interest “must be weighed as a ‘facto [r] in determining whether there is an abuse of discretion.’ ” Id. at 115 , 109 S.Ct. 948 (quoting Restatement (Second) of Trusts § 187, cmt. d (1959)).

We face a similar problem in the present case. As recognized by other circuit courts, the abuse of discretion standard under ERISA has yet to be developed enough to provide a consistent, “across-the-board test” for reviewing disability determinations. See, e.g., Donaho v. FMC Corp., 74 F.3d 894, 899 (8th Cir.1996). In particular, we do not believe that ERISA dictates that the standard should lack any degree of specific tailoring to the task of determining disability. Moreover, we reject the argument that Social Security disability determinations are distinguished by the fact that Congress and the Social Security Administration created an elaborate *1143 statutory and regulatory scheme governing such determinations.

In enacting ERISA, Congress also created an elaborate regime of laws and regulations governing covered benefit plans in order to protect the rights of participants and their beneficiaries. 29 U.S.C. § 1001 (b) (2000). A guiding principle such as the treating physician rule that is effective in helping plan administrators make fair and accurate disability determinations is consistent with this goal. Therefore, even assuming that the Social Security scheme is somehow more “elaborate” than ERISA, that alone is insufficient reason to decline to adopt the treating physician rule.

We note that two other circuits have declined to apply the treating physician rule to ERISA health benefits determinations. See Salley v. E.I. DuPont De Nemours & Co., 966 F.2d 1011, 1016 (5th Cir.1992) (stating in dicta that the court has “considerable doubt about holding the [treating physician] rule applicable in ERISA cases.”); Jett v. Blue Cross & Blue Shield of Ala., Inc., 890 F.2d 1137, 1140 (11th Cir.1989) (stating that even a plan’s failure to contact a treating physician is not an abuse of discretion); see also Sheppard v. Travelers Ins. Co., 32 F.3d 120, 126 (4th Cir.1994) (rejecting the treating physician rule when an ERISA plan is making a determination about the medical necessity of a prescribed treatment). It is worth emphasizing, however, that Salley, Jett and Sheppard all involved disputes over health care-rather than disability-benefits. This distinction is critical, given that the sole rationale for rejecting the rule advanced by these holdings is that a treating physician may be operating under a conflict of interest. See, e.g., Salley, 966 F.2d at 1016 (“[T]he treating physician would stand to profit greatly if the court were to find benefits should not be terminated.”) (citing Jett, 890 F.2d at 1140 ). Unlike health insurance benefits, which are paid directly to the treating physician, disability insurance benefits serve as a salary replacement payable to the employee. Thus, any potential conflict of interest in ERISA disability cases is no different from that which may exist in the Social Security context, where we have long found the treating physician rule to be applicable.

Indeed, far more troubling is the conflict of interest inherent when benefit plans repeatedly hire particular physicians as experts. Especially in cases such as this one, where the plan administrator is also the funding source, these experts have a clear incentive to make a finding of “not disabled” in order to save their employers money and to preserve their own consulting arrangements. None of the appellate courts deciding that the treating physician rule should not apply to health benefits determinations addressed the relevance of either potential conflict of interest.

Whereas differences exist between ERISA and Social Security in the discretion afforded plan administrators and ALJs in interpreting the terms of benefits coverage, we are not convinced that their roles differ significantly when it comes to deciding whether the facts of a particular case fall within clearly established definitions of what constitutes a disability. 7 As *1144 in the Social Security disability context, a rule requiring plan administrators to give special weight to the opinions of treating physicians is a similarly common sense requirement that, while inconsistent with the exercise of absolute discretion, is perfectly consistent with the plan administrator’s role in properly determining whether a particular claimant is disabled.

In the present case, given our need to remand for a determination as to whether the Delta Plan may have been operating under a conflict of interest, the scope of the plan administrator’s discretionary authority may not be so broad in the first place. Given the apparent existence of a conflict, we cannot agree that the issue of what standard to apply to Delta’s determination has been settled either by the parties’ stipulation or by the district court’s ruling. Therefore, we can make no final ruling as to whether the plan administrator abused its discretion when it terminated Regula’s disability benefits. Instead, we are properly concerned with the question of whether the district court improvidently ignored evidence that was before it at the time of summary judgment and that was material to the determination of an appropriate standard of review. For reasons discussed below, we conclude that the district court erred in this regard, and we remand for a proper determination as to the administrator’s impairment due to a conflict of interest.

B.

We review de novo the decision of a plan administrator to deny benefits “unless the benefit plan gives the administrator or fiduciary discretionary authority to determine eligibility for benefits or to construe the terms of the plan.” Firestone, 489 U.S. at 115 , 109 S.Ct. 948 ; see also Tremain v. Bell Industries, Inc., 196 F.3d 970 (9th Cir.1999). When the plan’s language confers such discretion, we review the decision of the administrator under an abuse of discretion standard. Tremain, 196 F.3d at 976 .

In this case, the Plan’s language appears to grant the Administrative Committee the broadest possible discretion in determining benefits eligibility:

The Administrative Committee also has “the exclusive power to interpret” the Plan and “its interpretation and decisions [are] final and conclusive.” Finally, the Administrative Committee is empowered to “decide all questions concerning the Plan.”

Nevertheless, the fact that the terms of the Plan vest the administrator with broad discretionary authority does not end our inquiry. In Firestone , the Supreme Court ruled that “if a benefit plan gives discretion to an administrator or fiduciary who is operating under a conflict of interest, that conflict must be weighed as a ‘factofr] in determining whether there is an abuse of discretion.’ ” Id. at 115 , 109 S.Ct. 948 ; see also Snow v. Standard Ins. Co., 87 F.3d 327, 331 (9th Cir.1996). We have held that our review in such cases is “still for abuse of discre *1145 tion, [but it] is ‘less deferential.’” Tre-main, 196 F.3d at 976 .

At the time of its ruling, the district court had before it Plan documents indicating that all of the members of the Administrative Committee were appointed by the Delta Board of Directors. Furthermore, although the benefit fund was organized as a trust, it was funded exclusively by Delta companies based on actuarial data. Thus, Delta effectively acted as both administrator and funding source for the Plan.

These factors formed the basis of our decision in Lang v. Long-Term Disability Plan of Sponsor Applied Remote Technology, Inc., supra, where we held that an insurer’s “conflict of interest, arising out of its dual role as the administrator and funding source for the Plan, affected its decision in Lang’s case.” Id. at 796; see also Tremain, 196 F.3d at 976-77 (finding a conflict of interest under similar circumstances). The Lang court approvingly cited the Eleventh Circuit’s decision in Brown v. Blue Cross & Blue Shield of Alabama, Inc., 898 F.2d 1556 (11th Cir.1990), ce rt. denied, 498 U.S. 1040 , 111 S.Ct. 712 , 112 L.Ed.2d 701 (1991), for the proposition that “plans such as this one, funded by insurers and also administered by them, are not true trusts.” Lang, 125 F.3d at 798 (citing Brown, 898 F.2d at 1567 ); see also Doe v. Group Hosp. & Med. Servs., 3 F.3d 80, 86-87 (4th Cir.1993) (“Even the most careful and sensitive fiduciary [when operating under a conflict of interest] may unconsciously favor its profit interest over the interests of the plan, leaving beneficiaries less protected than when the trustee acts without self-interest and solely for the benefit of the plan.”). Under such circumstances, plan benefits decisions are subject to a less deferential standard of review.

This “less deferential” standard consists of two steps:

Atwood v. Newmont Gold Co., Inc., 45 F.3d 1317, 1323 (9th Cir.1995). By providing material, probative evidence of a conflict, the plan beneficiary creates a rebut-table presumption that the plan’s decision was in fact a dereliction of its fiduciary responsibilities. The plan then “bears the burden of rebutting the presumption by producing evidence to show that the conflict of interest did not affect its decision to deny or terminate benefits.” Lang, 125 F.3d at 798 . If the plan fails to carry its burden, then we review de novo its decision denying benefits. Tremain, 196 F.3d at 976 .

Because in this case Delta acted as both administrator and funding source for the plan, and evidence of this conflict was before the district court at the time of summary judgment, the district court should have determined whether the apparent conflict of interest .was indeed serious enough to have resulted in a breach of fiduciary duty before choosing the appropriate standard of review to be applied to the Plan’s disability determinations. Thus, the court erred in failing even to consider whether Regula provided or could provide material, probative evidence of a breach of *1146 fiduciary duty. Instead the court simply concluded from the language of the Plan that an abuse of discretion standard applied, and then permitted the parties to enter into a stipulation presuming that this highly deferential standard was incontrovertibly the appropriate standard of review. Cf. Lang, 125 F.3d at 799-800 (“The district court did not conduct the appropriate conflict of interest analysis and hence accorded [the insurer] a deference to which it was not entitled.”) This conclusion was premature, and, as a result, this case has arrived for our review with an extremely unusual and infelicitous posture.

C.

We stated at the outset that, if the district court proceedings had not been curtailed by the parties’ stipulation, we may well have been able to determine the proper standard for the judicial review of the Plan’s termination of Regula’s disability benefits. However, under the present posture, such a determination by this court would be both premature and prejudicial, since the judicial assessment of an apparent conflict is managed through a burden-shifting scheme and the parties in this case, while presenting some relevant evidence for our review, have not been permitted the opportunity to make thoroughly responsive arguments regarding the fulfillment of their burdens. Still, we do not conclude that the district court must restart this process from scratch.

As stated above, a plan will be viewed as operating under an apparent conflict when it is both funded and administered by the insurer. In the present case, Delta both funds and administers the Plan. However, de novo review of a plan that gives discretion to an administrator remains inappropriate unless the plan beneficiary comes forward with material, probative evidence “tending to show that the fiduciary’s self-interest caused a breach of the administrator’s fiduciary obligations to the beneficiary.” Atwood, 45 F.3d at 1323 .

In Lang , we ruled that evidence of inconsistency in the administrator’s dealings with the beneficiary was material evidence of its self-interested behavior and grounds for ultimately reinstating benefits. The evidence of inconsistency in that case regarded a series of events in which the defendant plan first denied disability benefits because the beneficiary lacked evidence of a physical ailment and later, when confronted with an uncontroverted diagnosis by her treating physician that the beneficiary had fibromyalgia, continued to withhold benefits pending proof that the physical ailment alone was the cause of her disability. 125 F.3d at 799 ; see also Brown, 898 F.2d at 1569 (finding evidence of inconsistency in the administrator’s reversal of a payment decision for only one of two related claims on the basis of no new evidence). In Tremain v. Bell Industries, supra, we held that a plaintiff beneficiary established material, probative evidence of a conflict of interest where the plan administrator appeared to have relied upon an improper definition of disability in processing the beneficiary’s claim and where the administrator announced a determination of the beneficiary’s earning capacity for which it provided no supporting evidence. 196 F.3d at 977 .

Here, we find a similarly unsettling pattern of inconsistency and insufficiency in the plan administrator’s reasons for terminating the appellant’s benefits. First, the Plan’s sudden termination of Regula’s benefits came abruptly, with no evidence alleged of a significant change in his condition. This point is significant because the Plan had otherwise maintained the appellant’s long-term benefits for almost eight consecutive years, while reviewing his claim regularly at three month *1147 intervals. Second, and more importantly, once Regula appealed the Plan’s decision, the Plan based its final determination upon reports provided by examining physicians whose diagnoses contradicted those of Re-gula’s own treating physicians and to whom Regula had been referred by the Plan in response to the medical reports that Regula produced during his appeal. We note that the plan’s examining doctors contradicted the treating physicians on issues that the latter themselves had considered, and (with the single exception of the criticism made about Dr. Smith’s expertise) without providing specific reasons for their disagreement.

We cannot find that the Plan’s sudden and thinly supported departure from the prevailing diagnosis offered by Regula’s doctors was either consistent or sufficiently supported by the record. Furthermore, we find that, in light of the Plan’s apparent conflict of interest, the administrator’s decision to reject the opinions of the appellant’s treating physicians constitutes material, probative evidence of a conflict. On this point, we add deviation from the treating physician rule to the short list of factors by which a court may determine that an apparent conflict of interest has ripened into an actual, serious conflict, thereby permitting the court to engage in de novo review. Of course, the Plan could rebut this evidence by showing that its termination decision was supported by specific, legitimate reasons that are based on substantial evidence in the record. In this sense, our ruling resembles the treating physician doctrine found in the Social Security context. Cf. Morgan, 169 F.3d at 600 . However, because the district court concluded that the treating physician rule did not apply in the ERISA context, it did not provide the parties with an opportunity to complete this burden-shifting analysis.

We conclude that the proper standard of review to be applied to the plan administrator’s decision in this case can only be determined after an appropriate determination has been made in the district court regarding the Plan’s apparent compromise due to a conflict of interest. Therefore, we remand to the district court for a proper finding on this issue. On remand, we direct the district court to receive and consider additional evidence regarding Delta’s apparent conflict of interest, so as “to enable the full exercise of informed and independent judgment.” Mongeluzo v. Baxter Travenol Long Term Disability Ben. Plan, 46 F.3d 938 , 943 (9th Cir.1995). When examining the evidence for a conflict of interest, the district court is not limited to the administrative record before the plan administrator at the time that the benefits determination was made. Tremain, 196 F.3d at 977 . In addition, we note that, since we have already determined by our own review of summary judgment that the appellant did establish material, probative evidence of a conflict, the burden now falls upon the Plan to rebut the presumption that it was acting under a conflict.

VACATED AND REMANDED.

. As the district court pointed out, the physicians who examined Regula during this period included Drs. Michael Smith, H. Dale Richardson, Peter Schou, Alfred Bloch, Burton Wixen, Sandra Smith, and Dean H. Cummings. By Regula’s own admission, with the exceptions of Dr. Sandra Smith and Dr. Cummings, none of these physicians examined Re-gula within the four years prior to the Plan’s 1995 termination of his long-term disability benefits. A lengthy discussion of these superannuated reports is therefore unnecessary.

. The cases of Dannenberg and Cheng v. Commissioner, 878 F.2d 306 (9th Cir.1989), are not to the contrary. Both cases are distinguishable because they involved a district court's grant of partial summary judgment rather than the denial of summary judgment. See Dannenberg, 16 F.3d at 1074 ; Cheng, 878 F.2d at 308 . In both Dannenberg and Cheng , the parties stipulated to a final judgment on all claims, including those that were unaffected by the district court's grant of partial summary judgment. See Dannenberg, 16 F.3d at 1074 (stipulating to the dismissal of all claims including a Section 11 claim arising under the Securities Act of 1933 even though it was unaffected by the grant of partial summary judgment); Cheng, 878 F.2d at 308 (stipulating to the disallowance of all of a taxpayer’s claimed tax deductions even though some were left undisturbed by the district court’s grant of partial summary judgment). However, the unadjudicated claims could not properly be brought on appeal. See Dannenberg, 16 F.3d at 1076 ; Cheng, 878 F.2d at 310 . In contrast to Comsource, the proper procedure in Dannenberg and Cheng was for the parties to obtain Rule 54(b) certification from the district court because there were undecided claims not subject to appeal. The parties in each case failed to do so, and we properly dismissed their appeals to prevent piecemeal adjudication of their claims. See Dannenberg, 16 F.3d at 1078 ; Cheng, 878 F.2d at 310-11 .

. Rather, the statement by the parties that they wished, by entering the stipulation, "to preserve scarce resources” seems to indicate only that the stipulation was intended to reduce the public and private costs of further litigation at the district court level.

. See below at Section II.B. In this sense, the term defines not an absolute degree of deference but a mode of review, meaning that we review the determination of the decision maker to ascertain its reasonableness, according to the evidence before it at the lime that the determination was made, and without the benefit of hindsight. See, e.g., Taft v. Equitable Life Assurance Soc’y, 9 F.3d 1469 , 1472 (9th Cir.1993) (overruling a finding by the district court based on its review of evidence not before the plan administrator, because the review of such evidence under an abuse of discretion standard would lead "to the anomalous conclusion that a plan administrator abused its discretion by failing to consider evidence not before it”).

.In deciding this issue of first impression in this circuit, we are mindful of the fact that other circuits appear divided on this issue. *1140 The only circuit explicitly to rule on the issue has held that the treating physician rule does apply to ERISA disability cases. See Donaho v. FMC Corp., 74 F.3d 894, 901 (8th Cir.1996) ("We have held, in Social Security cases, that a reviewing physician’s opinion is generally accorded less deference than that of a treating physician ... and we apply this rule in disability cases under ERISA as well.") But see Delta Family-Care Disability & Survivorship Plan v. Marshall, 258 F.3d 834, 842-43 (8th Cir.2001) (distinguishing Donaho on the grounds that, although the record contained conflicting medical opinions regarding the beneficiary’s disability, the plan administrator did not abuse its discretion by terminating benefits where that decision was supported by substantial evidence). At least one other circuit has, in dicta, expressed doubts about the rule's applicability in the ERISA disability context. See Elliott v. Sara Lee Corp., 190 F.3d 601, 607-08 (4th Cir.1999) (concluding that it was "not persuaded" to apply the treating physician rule to the ERISA disability claim at issue, but determining that it need not reach the question because even if the rule applied, persuasive contradictory evidence existed to discount the opinion of the one treating physician who had expressed doubt as to whether the claimant could return to work).

. McKenzie v. General Telephone Co. of California, 41 F.3d 1310 (9th Cir.1994), is not to the contrary. McKenzie held that vocational evidence "is unnecessary where the evidence in the administrative record supports the conclusion that the claimant does not have an impairment which would prevent him from performing some identifiable job.” 41 F.3d at 1317 . McKenzie did not hold that vocational evidence would never be required, but simply that "the plan administrator is not required in every case where the 'any occupation' standard is applicable to collect vocational evidence in order to prove there are available occupations for the claimant.” Id. (emphasis added). Thus, McKenzie does not stand for the proposition that Social Security disability cases are fundamentally different from ERISA cases or that "extraneous” terms should not be read into ERISA plans.

. Social Security ALJs also have broad discretion in deciding whether the facts support a finding of disability. See Allen v. Heckler, 749 F.2d 577, 579 (9th Cir.1984) (“Where medical testimony is conflicting ... it is the ALJ’s role to determine credibility and to resolve the conflict.”); Allen v. Sec’y of Health & Human Servs., 726 F.2d 1470, 1473 (9th Cir.1984) ("It is the ALJ’s role to resolve evidentiary conflicts. If there is more than one rational interpretation of the evidence, the ALJ's conclusion must be upheld.”). At the same time, as with ALJs, the discretion of plan administrators is not unfettered. For example, plan *1144 administrators are required to obtain relevant medical evidence to guide their decisions. Salley v. E.I. DuPont de Nemours & Co., 966 F.2d 1011, 1015 (5th Cir.1992) (holding that the plan administrator abused its discretion by failing to obtain necessary medical information).