CExamEmpModels

---->{$r_i =\sum_{i=1}^{i-1} (r_{i} - (x_{i}+u_{i})) + (d_i-u_i)/2$}, ...,

->Given a quantity {$\theta$} that we're tyring to estimate, once we have an estimator {$\hat{\theta}$} and figured out its expected value and variance, we can obtain a {$1-\alpha$} ''confidence interval'' using the following formulas:
{$$1-\alpha \approx Prob \left(-z_{\frac{\alpha}{2}} \leq \frac{\hat{\theta}-\theta}{\sqrt{Var(\hat\theta)}} \leq z_{\frac{\alpha}{2}} \right)$$}

{$$1-\alpha \approx Prob \left(-z_{\frac{\alpha}{2}} \leq \frac{\hat{\theta}-\theta}{\sqrt{Var(\hat\theta)}}{}  \right)$$}

--> {$N = \text{# of times } x { was observed in the data.}$}

-->{$\displaystyle{Var[S_n(x)] \approx \frac{ (\text{# of observations } > x)(\text{(# of observations } <= x)}{n^3}} $}.

->which shows {$$S_n(x)$$} is unbiased.

{$$Var(S_n(x)) = \frac{1}{n^2} \cdot Var(Y) + t(x)^2Var(Z) + 2t(x) Cov(Y,Z),$$}

->(See discussion of the Covariance of 2 binomial RVs [[http://en.wikipedia.org/wiki/Binomial_distribution#Covariance_between_two_binomials|here]]).

{$E[S_n(x)] = S(x) \approx S_n(x); $}
-->{$E[S_n(x)] = S(x) \approx S_n(x); $}
-->{$E[S_n(x)] = S(x) \approx S_n(x);j $}

-->{$Cov(Y,Z) =E(YZ)-E(Y)E(Z) = - n^2 (1-S(c_{j-1}) (S(c_{j-1})-S(c_j)).$}

{$$Var(S_n(x)) = $$}

{$$E(S_n(x))= 1- \frac{1}{n} (E(Y) + t(x) E(Z)) $$}
{$$=  1- \frac{1}{n} \cdot \left[n (1-S(c_{j-1}) + n t(x) (S(c_{j-1})-S(c_j)) \right]$$}
{$$ = (1-t(x))S(c_{j-1})+t(x)S(c_j)$$;}

{$$=  1- \frac{1}{n} \cdot \left[n (1-S(c_{j-1}) $$}
{$$ + n t(x) (S(c_{j-1})-S(c_j)) \right] = (1-t(x))S(c_{j-1})+t(x)S(c_j)$$;}

{$$=  1- \frac{1}{n} \cdot \left[n (1-S(c_{j-1})  + n t(x) (S(c_{j-1})-S(c_j)) \right] = (1-t(x))S(c_{j-1})+t(x)S(c_j)$$;}

{$$what??$$}

{$$E(S_n(x)) = 1- \frac{1}{n} (E(Y) + t(x) E(Z)) =  1- \frac{1}{n} \cdot \left[n (1-S(c_{j-1})  + n t(x) (S(c_{j-1})-S(c_j)) \right] = (1-t(x))S(c_{j-1})+t(x)S(c_j)$$;}

{$$E(S_n(x)) = 1- \frac{1}{n} (E(Y) + t(x) E(Z)) =  1- \frac{1}{n} \cdot \left[n (1-S(c_{j-1})  + n t(x) (S(c_{j-1})-S(c_j)) \right] = (1-t(x))S(c_{j-1}+t(x)S(c_j)$$;}

-->{$E[S_n(x)] = S(x) \approx S_n(x); $}

{$$E(S_n(x) = 1- ]frac{1}{n} (E(Y) + t(x) E(Z)) =  1
- \frac{1}{n} \cdot \left[n (1-S(c_{j-1})  + n t(x) (S(c_{j-1})-S(c_j)) \right] = (1-t(x))S(c_{j-1}+t(x)S(c_j)$$;}

{$$E(S_n(x) = 1- E(Y) + t(x) E(Z) =  1- \frac{1}{n} \cdot n(1-S(c_{j-1}) + n t(x) (S(c_{j-1})-S(c_j)) (1-t(x))S(c_{j-1}+t(x)S(c_j) = (1-t(x))S(c_{j-1}+t(x)S(c_j)$$;}

->It's not possible to calculate {$E[\hat{q_2}]$} and {$Var[\hat{q_2}]$}, because {$X = 0$} would make the denominator zero; an alternative is to calculate the conditional variance given, say, {$X=k$}.  Note that we can estimate the resulting RV, {$Y/k$}, with a new empirical estimate {$S_k(x) = \text{# of deaths between 2 and 3}/k$}, where is Binomial with parameters {$k$} and {$(S(2)-S(3))/S(2)$}, so that as the case above,

showing that the estimate is unbiased.  Note that
{$$\hat{q_2}=\frac{\text{# of deaths between 2 and 3}}{k}. $$}

{$$E(S_n(x) = 1- E(Y) + t(x) E(Z) =  1- \frac{1}{n} \cdot n(1-S(c_(j-1)) + n t(x) (S(c_{j-1})-S(c_j)) (1-t(x))S(c_{j-1}+t(x)S(c_j) = (1-t(x))S(c_{j-1}+t(x)S(c_j)$$;}

{$$S_n(x) = 1 - \frac{1}{n} Y + t(x)Z,$$}

{$$E(S_n(x) = 1- E(Y) + t(x) E(Z) = (1-t(x))S(c_{j-1}+t(x)S(c_j)  (1-t(x))S(c_{j-1}+t(x)S(c_j)$$;}

{$$E[S_n(x)] = \frac{1}{n} E[Y] = \frac{1}{n} (n S(x)) = S(x) \approx S_n(x); $$}

{$$E[\hat{q_2}| \text{# of people alive at }2 = k] = E[Y/k] = \frac{1}{k}\frac{k(S(2)-S(3))}{S(2)}\approx  \frac{S_n(2) - S_n(3)}{S_n(2)}=\frac{\text{# of deaths between 2 and 3}}{k}; $$}

-->c.  If we have ogives and histograms (i.e., the data is grouped), {$E[S_n(x)] = S(x) \approx S_n(x); $}

{$$E[\hat{q_2}| \text{# of people alive at }2 = k] = E[Y/k] = \frac{1}{k}\frac{k(S(2)-S(3))}{S(2)}\approx  = \frac{S_n(2) - S_n(3)}{S_n(2)}=\frac{\text{# of deaths between 2 and 3}}{\text{# of people alive at 2}}; $$}
{$$Var[\hat{q_2} | \text{# of people alive at }2 = k] = Var[Y/k] \approx \frac {S_k(x)(1-S_k(x)}{k}=Var[\hat{q_2} | \text{# of people alive at }2 = k] \approx \frac{\text{(# of deaths between 2 and 3)(}k -\text{# of deaths between 2 and 3)}}{k^3}. $$}

-->{$E[\hat{q_2}| \text{# of people alive at }2 = k] = E[Y/k] = \frac{1}{k}\frac{k(S(2)-S(3))}{S(2)}\approx  = \frac{S_n(2) - S_n(3)}{S_n(2)}=\frac{\text{# of deaths between 2 and 3}}{\text{# of people alive at 2}}; $}
{$$Var[\hat{q_2} | \text{# of people alive at }2 = k] = Var[Y/k] \approx \frac {S_k(x)(1-S_k(x)}{k};$$}
->This is the same as
{$$\displaystyle{Var[\hat{q_2} | \text{# of people alive at }2 = k] \approx \frac{\text{(# of deaths between 2 and 3)(}k -\text{# of deaths between 2 and 3)}}{k^3}}. $$}

->b.  Empirical estimate of (say) {$q_2 = \frac{S(3) - S(2)}{S(2)}$} is {$\hat{q_2}=\frac{S_n(3) - S_n(2)}{S_n(2)} = \frac{Y}{X}$}, where

->It's not possible to calculate {$E[\hat{q_2}]$} and {$Var[\hat{q_2}]$}, because {$X = 0$} would make the denominator zero; an alternative is to calculate the conditional variance given, say, {$X=k$}.  Note that we can estimate the resulting RV, {$Y/k$}, with its own empirical estimate {$S_k(x) =Y/k$}, so that as the case above,
-->{$E[\hat{q_2}| \text{# of people alive at }2 = k] = E[Y/k]\approx \frac{S_k(x)}{k} = \frac{\text{# of deaths between 2 and 3}}{\text{# of people alive at 2}}; $}

-->{$E[S_n(x)] = S(x) \approx S_n(x); $}
-->{$Var[S_n(x)] = \frac{1}{n^2} Var[Y] = S(x)(1-S(x))/n \approx S_n(x)(1-S_n(x))/n; $}

-->{$Var[\hat{q_2} | \text{# of people alive at }2 = k] = Var[Y/k] \approx \frac {S_k(x)(1-S_k(x)}{k}$};

-->{$\displaystyle{Var[\hat{q_2} | \text{# of people alive at }2 = k] \approx \frac{\text{(# of deaths between 2 and 3)(}k -\text{# of deaths between 2 and 3)}}{k^3}} $}.

-->{$\displaystyle{{$Var[S_n(x)] \approx \frac{\text{(# of observations } > x)(\text{# of observations } <= x)}{n^3}} $}.

-->{$E[\hat{q_2}| \text{# of people alive at }2 = k] = E[Y/k]\approx \frac{S_k(x)}{k} = \frac{\text{# of deaths between 2 and 3}}{\text{# of people alive at 2}}}; $}

->{$E[\hat{q_2}] = ?? \approx S_n(x); $}
->It's not possible to calculate {$Var[\hat{q_2}]$}, because {$X = 0$} would make the denominator zero; an alternative is to calculate the conditional variance given, say, {$X=k$}.  Note that we can estimate the resulting RV, {$Y/k$}, with its own empirical estimate {$S_k(x) =Y/k$}, so that as the case above,

-->{$\displaystyle{\frac{\text{(# of observations } > x)(\text{# of observations } <= x}{n^3}} $}.

->This is {$\displaystyle{\frac{\text{(# of deaths between 2 and 3)(}k -\text{# of deaths between 2 and 3)}}{k^3}} $}.

->Given a complete data set of {$n$} points, the empirical estimate of {$S(x)$} is {$S_n(x)= Y/n$},

-->Empirical estimate of (say) {$q_2 = \frac{S(3) - S(2)}{S(2)}$} is {$\hat{q_2}=\frac{S_n(3) - S_n(2)}{S_n(2)} = \frac{Y}{X}$}, where
--->{$X$} = # of people alive at duration 2,
--->{$Y$} = # of deaths between duration 2 and 3. 
-->{$E[\hat{q_2}] = ?? \approx S_n(x); $}
-->It's not possible to calculate {$Var[\hat{q_2}]$}, because {$X = 0$} would make the denominator zero; an alternative is to calculate the conditional variance given, say, {$X=k$}.  Note that we can estimate the resulting RV, {$Y/k$}, with its own empirical estimate {$S_k(x) =Y/k$}, so that as the case above,
--->{$Var[\hat{q_2} | \text{# of people alive at }2 = k] = Var[Y/k] \approx \frac {S_k(x)(1-S_k(x)}{k}$};
-->This is {$\displaystyle{\frac{\text{(# of deaths between 2 and 3)(}k -\text{# of deaths between 2 and 3)}}{k^3}} $}

--->{$Var[\hat{q_2} | \text{# of people alive at }2 = k] = Var[Y/k] \approx \frac {S_k(x)(1-S_k(x)}{k} = \frac{\text{(# of deaths between 2 and 3)(}k -\text{# of deaths between 2 and 3)}}{k^3} $}

--->{$Var[\hat{q_2 | \text{# of people alive at }2 = k} | ] = Var[Y/k] \approx \frac {S_k(x)(1-S_k(x)}{k} = \frac{\text{(# of deaths between durations 2 and 3)(}k -\text{# of deaths between durations 2 and 3}}{k^3} $}

-->{$Var[S_n(x)] = S(x)(1-S(x))/n \approx S_n(x)(1-S_n(x))/n; $}

-->Empirical estimate of (say) {$q_2 = \frac{S(3) - S(2)}{S(2)}$} is {$\hat{q_2}=\frac{S_n(3) - S_n(2)}{S_n(2)} = \frac{Y}{n-X}$}, where
--->{$X$} = # of deaths between duration 0 and 2,
--->{$Y$} = # of deaths between duration 2 and 3.

-->It's not possible to calculate {$Var[\hat{q_2}]$}, because {$X = 30$} would make the denominator zero; an alternative is to calculate the conditional variance
--->{$Var[\hat{q_2} | X= 29 ]$ = Var[] S(x)(1-S(x))/n \approx S_n(x)(1-S_n(x))/n; $}

-->It's not possible to calculate {$Var[\hat{q_2}]$}, because {$X = 30$} would make the denominator zero; an alternative is to calculate

-->Given a complete data set of {$n$} points, the empirical estimate of {$S(x)$} is {$S_n(x)= Y/n$},

-->Empirical estimate of (say) {$q_2)$} is {$\hat{q_2}=\frac{S_n(3) - S_n(2)}{S_n(2)} = \frac{Y}{n-X}$}, where

-->{$E[\hat{q_2}] = S(x) \approx S_n(x); $}
-->{$Var[S_n(x)] = S(x)(1-S(x))/n \approx S_n(x)(1-S_n(x))/n; $}

->Given a complete data set with {$n$} data points with some unknown survival function {$S(x)$}, we can define the empirical estimator {$S_n(x)$} as above; one way to see this is
-->{$S_n(x) = Y/n$},

-->Empirical estimate of (say) {$q_2)$} is {$\frac{S_n(3) - S_n(2)}{S_n(2)}$};
-->{$E[S_n(x)] = S(x) \approx S_n(x); $}

->Given a complete data set with {$n$} data points with some unknown survival function {$S(x)$}, we can define the empirical estimator {$S_n(x)$} as above; then

->where {$Y$} is the # of observations that are greater than {$x$}.  Then ''{$Y$} is a binomial distribution with parameters {$n$} and {$S(x)$}.''

--->single-decrement mortality probabilities {$q_{j}^{'(d)}$}, by treating the death counts as uncensored observations and withdrawls as cencored observations;
--->single-decrement withdrawl probabilities {$q_{j}^{'(w)}$}, by treating the withdrawl counts as uncensored observations and deaths as cencored observations.

--->{$y_j = \frac{S(c_j)-S(c_{j+1}}{S(c_j)} = \frac{x_j}{r_j} $}.

-->'''Large Data Sets''':  Sometimes the data is so large that it's not reasonable to keep track of all the {$y_j$}'s.  So instead we have intervals of interest:  Let {$c_o < c_1 < \ldot < c_k$} be boundaries of such intervals. 

---> {$d_i$} = # of truncated observations where the truncation point is in {$[c_{i-1}, c_{i})$};

-->I.  All truncation occurs at the beginning of the interval, and all censoring occurs at the end of the interval. 
-->(For example, all lives enter and leave study using their insuring age.)

-->Define the risk set to be
--->{$r_1 = d_1$},
--->{$r_2 = r_1 - (x_1+u_1) + d_2$}, ...,
--->{$r_i = r_{i-1} - (x_{i-1}+u_{i-1}) + d_i$}, ...,
-->II.  Truncation and censoring occur uniformly throughout the interval, and the uncensored observations occur in the middle of the interval.
-->(For example, more detailed data gathering would invariably give something closer to this scenario than the former.)

-->Define the risk set to be
--->{$r_1 = (d_1-u_1)/2$},
--->{$r_2 = d_1 - (x_1+u_1) + (d_2-u_2)/2$}, ...,
--->{$r_i =\sum_{i=1}^{i-1} (r_{i} - (x_{i}+u_{i})) + (d_i-u_i)/2$}, ...,

--->

-->Large Data Sets:  Sometimes the data is so large that it's not reasonable to keep track of all the {$y_j$}'s.  So instead we have intervals of interest:  Let {$c_o < c_1 < \ldot < c_k$} be boundaries of such intervals.  For {$i = 1, \ldots, n,$} let

-->Then the risk set is

Example:  Mortality study on a large population; {$c_j$}'s can be integer ages.

---> {$u_i$} = censored value for that observation.