Friedland15.Evaluation

Reading: Friedland, J.F., Estimating Unpaid Claims Using Basic Techniques, Casualty Actuarial Society, Third Version, July 2010. The Appendices are excluded.

Chapter 15: Evaluation of Methods

Pop Quiz

Ian-the-Intern has a collision deductible of $100 and yesterday he backed into a fire hydrant causing $600 worth of damage to his car. His auto insurer has a 60% quota-share reinsurance treaty. What is his insurer's loss net of all deductibles and reinsurance? Click for Answer 

Study Tips

VIDEO: F-15 (001) Evaluation → 7:00 Forum

This is a really great chapter because it ties together concepts from all previous chapters. The only thing that's totally new is a formula for calculating expected emergence. There are 2 versions and they are both pretty easy: (There is a web-based problem in BattleQuiz 2 so you can drill yourself on the emergence calculation.)

• expected emergence of reported losses
• expected emergence of paid losses

Most of the exam questions from this chapter ask you to do one or more of the following:

• evaluate the accuracy of an estimate from a given reserving method
• identify a scenario that explains:

→ differences in estimates between different accident years

→ differences in estimates between different reserving methods

→ changes in estimates for a given reserving method between successive evaluation dates

• suggest an adjustment or alternate method to improve accuracy of estimates

The best way to get the hang of answering these types of questions is to go through all old exam problems. There are a lot but don't let that scare you – many of them don't take long to do. It's a heavily tested chapter and is an excellent way to review concepts related the reserving methods from previous chapters.

Estimated study time: 1 week (not including subsequent review time)

BattleTable

Based on past exams, the main things you need to know (in rough order of importance) are:

• emergence

- calculate expected emergence for paid or reported claims or claim counts, with interpolation if necessary

- compare expected emergence to actual emergence and decide whether to revise estimate of ultimate

• impact of operational and external changes on reserving methods and estimates of ultimates
• assess validity of reserve estimates or methods in various situations
• identify scenario that is consistent with given information or explains differences between methods

reference	part (a)	part (b)	part (c)	part (d)
E (2019.Fall #25)	emergence: - of losses	emergence: - actuary's decision
(2018.Spring #24)	BattleActs PowerPack
(2018.Spring #26)	BattleActs PowerPack
E (2017.Fall #27)	assess method: - paid & rptd devlpt	assess method: - Freq-Sev, rptd BF	assess method: - paid devlpt, paid BF
E (2017.Fall #28)	assess CDFs: - devlpt, BF, Freq-Sev	assess ECR: - BF method
E (2017.Spring #21)	identify scenario: - consistent with results	identify scenario: - to explain difference	alternate method: - for part (a)	alternate method: - for part (b)
E (2017.Spring #26)	assess ECR: - BF method	method comparison: - to Benktander	investigating results: - questions for mgmt 1
E (2016.Fall #16)	Friedland05.Triangles	Friedland05.Triangles	Friedland06.Diagnostics	select method: - settlement rate change
E (2016.Fall #18)	BF ECR: - calculate	select method: - calculate unpaid	assess method: - ECR, paid BF, paid devlpt
E (2016.Fall #27)	emergence: - counts	emergence: - limitations
E (2016.Spring #24)	emergence: - losses 2	emergence: - actuary's decision
E (2016.Spring #25)	identify scenario: - to explain difference	identify scenario: - to explain difference
E (2015.Fall #19)	assumptions: - recommend changes	revise: - estimates	assess: - revised estimates
E (2015.Fall #22)	impact / solution: - long development pattern	impact / solution: - tort reform	impact / solution: - higher deductibles	impact / solution: - faster claims processing
E (2015.Fall #25)	emergence: - losses based on devlpt	emergence: - losses based on selected	emergence: - actuary's decision
E (2014.Fall #19)	impact / solution: - increasing case strength	impact / solution: - premium growth	impact / solution: - higher limits
E (2014.Fall #22)	emergence: - of losses	emergence: - linear interpolation issues
E (2014.Fall #24)	select method: - that will be accurate	select method: - that will over-estimate	select method: - that will under-estimate
E (2014.Spring #17)	impact (case strength): - on rptd development	impact (case strength): - on ECR method	impact (case strength): - on rptd BF method	impact (case strength): - on Cape Cod method
E (2014.Spring #22)	emergence: - of losses	emergence: - of losses	identify scenario: - re: actuary's decision	identify scenario: - re: actuary's decision
E (2014.Spring #23)	cumulative % rptd: - calculate	Friedland12.CaseOS
E (2013.Fall #18)
E (2013.Fall #24)
E (2013.Spring #21)
E (2013.Spring #22)
E (2013.Spring #26)

¹ See Friedland04.Meetings for potential questions.

² See this forum discussion for a potentially faulty reasoning presented in the examiner's report solution to part (a).

Full BattleQuiz

In Plain English!

Selecting a Final Estimate

Suppose you get to the end of your reserve analysis as of Dec 31, 2025 and have estimates of ultimates from several different methods:

AY	Paid Development	Reported Development	Paid Berquist-Sherman	Reported Berquist-Sherman
2020	1,000	1,000	1,000	1,000
2021	1,000	1,000	1,000	1,000
2022	1,000	1,000	1,000	1,000
2023	1,000	1,050	1,000	1,000
2024	1,000	1,125	1,000	1,000
2025	1,000	1,300	1,000	1,000

Here are some typical questions:

• identify a scenario that explains differences in estimates between different accident years
• identify a scenario that explains differences in estimates between different reserving methods

To answer these, you need to make a few observations:

• paid development and both Berquist-Sherman methods show an estimate of 1,000 for all AYs
• reported development starts the same as the others but begins increasing at AY 2023

Let's consider the following potential explanations for a rise in the estimate for reported development:

• change in loss ratio
• change in settlement rate of claims
• change in case reserve strength
• change in mix of business
• large loss or losses in AYs 2023, 2024, 2025
• court decision
• change in development pattern by some other cause (this is a catch-all because a change in development pattern can be caused by many different things)

It probably isn't a change or deterioration in the loss ratio because that would have affected the paid development method also. You have to pick something that would specifically affect only reported development. The best option from this list is a change (increase) in case reserve strength. We know from previous chapters that an increase in case strength causes reported development to over-estimate. The reason, of course, is that historical LDFs were taking lower case reserves to ultimate so if we suddenly increase case strength, these historical LDFs will overshoot.

Side note: If I asked you when the increase in case strength occurred, the answer is not CY 2023. You know that because an increase in CY 2023 would affect unreported claims from AY 2023 and prior, but AY 2022 is accurate. (How far back the effect goes depends on the length of the development tail.) Actually, you can't tell from the given data exactly when the case strengthening occurred, but a better guess would be the beginning of CY 2025. All AY 2025 claims would then be reserved at the higher level. The effect would be less noticeable for AY 2024 because only claims reported after 12 months would be at the higher level. (AY 2024 claims reported within the first 12 months would be reserved at the lower level so the distortion is less pronounced for AY 2024 versus AY 2025.) Similarly, even fewer claims from AY 2023 would still be unreported and receive the higher case reserves at the beginning of CY 2025. Since AY 2022 is accurate, it appears that all AY 2022 claims were reported by the change in case reserve adequacy, whenever it might have been.

But getting back to the original questions: The scenario that explains the differences in estimates is most likely an increase in case reserve adequacy:

• it explains the increase in the reported development estimates starting in AY 2023
• it explains why paid development and paid Berquist-Sherman are accurate (an increase in case strength doesn't affect paid data)
• it explains why the reported Berquist-Sherman method is accurate for all AYs (reported BS corrects for changes in case strength)

I made my example very simple to illustrate the point. Below is a harder exam problem for you to try. In an exam situation you would only have about 8 minutes to solve it and write out your answer, but for learning it's worth spending a couple of hours. You may have to go back and review details of different methods from previous chapters.

E (2016.Spring #25)

Here are a few more similar exam problems in the quiz.

mini BattleQuiz 1

Emergence Patterns

Here's the emergence formula we're going to use in this section:

expected reported loss (m, m+12)    =   ( current unreported ) m   x   [ ( %reported m+12  –  %reported m )  /  %unreported m ]

If m represents development months then the left side of the formula represents the dollars expected to be reported in the development interval m to m + 12. Let's look at an example to see how this works. Suppose you've applied the development method to a triangle of reported losses as of Dec 31, 2025:

• AY 2025 reported loss ₁₂ = 96
• CDF_{12 → ult} = 2.50
• CDF_{24 → ult} = 1.50

The estimated ultimate loss is obviously 96 x 2.50 = 240

Question: How could you assess the accuracy of your estimate?

Solution: There are several ways of doing this:

• test assumptions of the method (if assumptions are satisfied then our confidence is increased)
• calculate metrics such as loss ratio, severity, frequency, pure premium (if the metrics are within a reasonable range then confidence is increased)
• compare to reported development estimates obtained at successive valuation dates (if estimates are close then our confidence is increased)
• compare to other methods (if other methods produce a similar result then our confidence is increased)
• perform a true retroactive test (wait until all claims for AY 2025 are closed at which point we know the true ultimate, but you might have to wait several years)
• calculate expected emergence in future periods and compare to actual emergence (this would be a partial retroactive test)

In this discussion, we're going to use our formula to calculate the expected emergence of reported losses from 12 to 24. Recall:

• %reported₁₂ = 1 / CDF₁₂ = 40.0%
• %reported₂₄ = 1 / CDF₂₄ = 66.7%

By simple logic:

•  %unreported₁₂  =  1 – %reported₁₂  =  1 – 40.0%  =  60.0%

We also have:

• ( current unreported ) ₁₂   =   ultimate – reported₁₂   =   240 – 96 = 144

Substituting into our formula using the above values with m = 12 gives:

• (expected reported loss)_{(12, 24)}

= 144 x ( 66.7% - 40.0% ) / 60.0%

= 64.1

I used the label expected reported loss but this is the same as expected emergence. If we wait 1 year and find the actual emergence is something reasonably close to 64.1 then our confidence increases. If not, we may need to investigate the reason for the discrepancy and potentially change our estimate of ultimate.

There's also an analogous formula for paid emergence:

expected paid loss (m, m+12)    = ( current unpaid ) m   x   [ ( %paid m+12  –  %paid m )  /  %unpaid m ]

Now try part (a) of this exam problem:

E (2016.Fall #27)

Part (b) of the problem is a memory question asking about potential limitations of calculating expected emergence for comparison with actual emergence. Here are some possible answers:

• paid and reported losses are generally skewed towards beginning of a time period year because %paid and %reported curves are generally concave downward functions (annual time frames may not produce accurate results)

→ use a shorter time frame such as quarterly     (thx CG!)

• if the estimate of ultimate is not based on a development method then the CDFs from a development method may not be appropriate in the formula

→ divide paid or reported values by the given estimate of ultimate to get %paid and %reported instead of just inverting the CDFs     (thx CG!)

• formula doesn't account for operational changes since projected emergence is based on historical patterns (Exs: change in mix of business, settlement rate, case strength)

→ calculate emergence of counts since counts may be less distorted by operational changes

Interpolation: Suppose in the above problem, we wanted the expected emergence in the period from Dec 31, 2025 only to Mar 31, 2026. Using linear interpolation, we just divide the annual emergence by 4. The result for our example is 64.1/4 = 16.0. This assumption of uniform emergence over the year is likely not correct however. The true emergence is probably something greater than 16.0 for January through March and something less than that for October through December but this isn't discussed any further in the source text.

Here's a problem involving interpolation:

E (2019.Fall #25)

Here a a couple of practice problems on paid emergence and a Quick-Vid on the corresponding web-based version of the problem in the next quiz:

Practice: 2 paid emergence problems

Quick-Vid: F-15 (010) Emergence ~ 1:00

And the quiz...which includes a few exam problems and a web-based problem for practicing the emergence formula as many times as you want with random numbers...

mini BattleQuiz 2

Retroactive Testing

I mentioned retroactive testing above as a way of assessing the accuracy of your reserve estimate. For us, retroactive testing means calculating expected emergence and comparing it to actual emergence. There isn't much more to say about it, but there is a short section in Friedland that relates to part (b) of this exam problem:

E (2016.Spring #24)

Part (b) is conceptual but let's think through it with the numerical example of expected emergence from earlier. Just to summarize, this is what we had:

AY 2025 reported loss 12	96
AY 2025 estimated ultimate	240
AY 2025 unreported loss 12	144
AY 2025 emergence(12, 24)	64.1

From this, we expect the AY 2025 unreported amount at 24 months to be = 144 - 64.1 = 79.9 which is the expected IBNR at 24 months. Let's suppose we now wait until the end of CY 2026 and you find the actual emergence is 90, quite a bit higher than the expected. What would you do?

• reduce the IBNR, potentially by the difference 90 - 64.1 = 25.9 (because your investigation concludes reporting has been simply been accelerated – no change in ultimate)

→ new IBNR for AY 2025 at 24 months = 79.9 - 25.9 = 54

• leave the IBNR unchanged (because your investigation concludes this is just random variation, possibly a large loss, and claims are otherwise reported as per usual, so the implied ultimate increases by the amount of random variation to 240 + 25.9 = 265.9)

→ IBNR for AY 2025 at 24 months = 79.9

• increase the IBNR by a some factor in proportion to the excess emergence (because your investigation concludes that overall loss experience is indeed worse and will continue to be worse than expected)

→ new IBNR for AY 2025 at 24 months = 79.9 x (appropriate judgmental factor)

This is essentially the question posed by Hugh White and his answer is that any of the 3 actions can be reasonable depending on circumstances. The exam problem alludes to these 3 possibilities, but goes a little further in that it asks you to identify a reserving method that would achieve each of these 3 outcomes.

mini BattleQuiz 3

Extra Exam Problems

Just some more exam problems since this is a heavily tested chapter...

mini BattleQuiz 4

Full BattleQuiz

POP QUIZ ANSWERS

• The insurer's loss net of Ian's deductible is $500.
• After the 60% quota-share reinsurance treaty is applied, the insurer's loss net of deductibles and reinsurance = 500 x 60% = $300

Go back