Tag Archive for laboratory

Validating molecular assay using R

Validation of Hepatitis B viral count using R-computing

Validation of Hepatitis B viral count using R-computing

This is an example on how R-computing can be used for validation of an quantitative assay. In this case two assays for Hepatitis B viral count are compared.

## Loading required package: seriation
## Loading required package: nlme

In a summary. 'Zero' values have been changed to '1' in order to be able to plot in logaritmic scale. The lower limit of detection (LLD) at home-lab is 10 IU/ml and the LLD at the reference-lab os 20 IU/ml. So, if the result is <20IU/ml, the detected value could be anywhere between 1 and 20. Therefore, the lower limit of detection has been set for home-lab at '5 IU/ml' and '10 IU/ml' for the reference lab.

##       PIN              Ref_lab            Home_lab       
##  Min.   :14091022   Min.   :1.00e+00   Min.   :1.00e+00  
##  1st Qu.:14104055   1st Qu.:2.24e+02   1st Qu.:6.39e+02  
##  Median :14121724   Median :1.98e+03   Median :2.17e+03  
##  Mean   :14116291   Mean   :1.64e+07   Mean   :2.15e+07  
##  3rd Qu.:14132019   3rd Qu.:1.52e+05   3rd Qu.:8.42e+05  
##  Max.   :14132394   Max.   :1.70e+08   Max.   :2.88e+08
##        PIN Ref_lab Home_lab
## 1 14091022       1      184
## 2 14091023    3473     3473
## 3 14104024    2976     2558
## 4 14104025     988     1001
## 5 14104026   96670 20892951
## 6 14104141 1526000  1048129

To make it more easy, the set of values from Reference-lab = 'x'. The set of values from Home-lab = 'y'

Calculate the means and difference between the two sets (x and y)

# derive difference
## [1] 16447938
## [1] 21548265
# mean Ref_lab - mean Home_lab
## [1] -5100327

Because n=17 is small, the distribution of the differences should be approximately normal. Check using a boxplot and QQ plot. There is some skew.

HepB_Web$diff <- x-y
##  [1]       -183          0        418        -13  -20796281     477871
##  [7]         77   12039815       -176   34930655 -118402140       -282
## [13]         -9     -53972       -171      -1757          0        265

plot of chunk Boxplot


plot of chunk Boxplot
Shaphiro test of normality.

##  Shapiro-Wilk normality test
## data:  HepB_Web$diff
## W = 0.479, p-value = 5.294e-07

The normality test gives p < 0.003, which is small, so we
reject the null hypothesis that the values are distributed normally.

This means that we cannot use the student t-test. Instead, use the Mann-Whitney-Wilcoxon Test. We can decide whether the population distributions are identical without assuming them to follow the normal distribution.

wilcox.test(x, y, paired = TRUE)
## Warning: cannot compute exact p-value with zeroes
##  Wilcoxon signed rank test with continuity correction
## data:  x and y
## V = 59, p-value = 0.6603
## alternative hypothesis: true location shift is not equal to 0

p > 0.05 and therefore the H0 is NOT rejected.
The two populations are identical.

Just to see what happens in the Student T-test.
A paired t-test: one sample, two tests
H0 = no difference; H1 = mean of 2 tests are different
mu= a number indicating the true value of the mean
(or difference in means if you are performing a two sample test).

t.test(x, y, mu=0, paired=T, alternative="greater")
##  Paired t-test
## data:  x and y
## t = -0.7202, df = 17, p-value = 0.7594
## alternative hypothesis: true difference in means is greater than 0
## 95 percent confidence interval:
##  -17420746       Inf
## sample estimates:
## mean of the differences 
##                -5100327

p = 0.759. Because p is larger than alpha, we do NOT reject H0.
In other words, it is unlikely the observed agreements happened by chance.
However, because the populations do not have a normal distribution, we can not use the outcome if this test.

For correlation, three methods are used: pearson, kendall and spearman at a confidence level of 95%.

# correlation of the two methods
cor.test(x, y, 
         alternative = c("two.sided", "less", "greater"),
         method = c("pearson", "kendall", "spearman"),
         exact = NULL, conf.level = 0.95)
##  Pearson's product-moment correlation
## data:  x and y
## t = 11.19, df = 16, p-value = 5.646e-09
## alternative hypothesis: true correlation is not equal to 0
## 95 percent confidence interval:
##  0.8472 0.9784
## sample estimates:
##    cor 
## 0.9416

The correlation with the spearman test is 0.9416175. Almost perfect correlation.

Plotting the two methods using logarithmic scales.

g <- ggplot(HepB_Web, aes(log(Home_lab), log(Ref_lab)))

# add layers
g + 
  geom_smooth(method="lm", se=TRUE, col="steelblue", size = 1) +
  geom_point(size = 3, aes(colour = x)) +
  scale_colour_gradient("IU/ml", high = "red", low = "blue", space = "Lab") +
  labs(y = "Reference lab (log IU/ml)") +
  labs(x = "Home lab (log IU/ml)") +
  theme_bw(base_family = "Helvetica", base_size = 14) +

plot of chunk Plotting
Summary data on the correlation line.

regmod <- lm(y~x, data=HepB_Web)
## Call:
## lm(formula = y ~ x, data = HepB_Web)
## Residuals:
##       Min        1Q    Median        3Q       Max 
## -76044958   1901358   1905277   1905580  47898082 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) -1.91e+06   5.98e+06   -0.32     0.75    
## x            1.43e+00   1.27e-01   11.19  5.6e-09 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## Residual standard error: 23800000 on 16 degrees of freedom
## Multiple R-squared:  0.887,  Adjusted R-squared:  0.88 
## F-statistic:  125 on 1 and 16 DF,  p-value: 5.65e-09

The Bland-Altman Analysis. To check if there is a bias.

##      Ref_lab  Home_lab       diff
## 1          1       184       -183
## 2       3473      3473          0
## 3       2976      2558        418
## 4        988      1001        -13
## 5      96670  20892951  -20796281
## 6    1526000   1048129     477871
## 7        919       842         77
## 8   23250000  11210185   12039815
## 9        421       597       -176
## 10 101000000  66069345   34930655
## 11 170000000 288402140 -118402140
## 12       483       765       -282
## 13         1        10         -9
## 14    169800    223772     -53972
## 15       158       329       -171
## 16        22      1779      -1757
## 17         1         1          0
## 18     10970     10705        265
BlandAltman(x, y,
            x.name = "Reference lab IU/ml",
            y.name = "Home lab IU/ml",
            maintit = "Bland-Altman plot for HBV count",
            cex = 1,
            pch = 16,
            col.points = "black",
            col.lines = "blue",
            limx = NULL,
            limy = NULL,
            ymax = NULL,
            eqax = FALSE,
            xlab = NULL,
            ylab = NULL,
            print = TRUE,
            reg.line = FALSE,
            digits = 2,
            mult = FALSE)
## NOTE:
##  'AB.plot' and 'BlandAltman' are deprecated,
##  and likely to disappear in a not too distant future,
##  use 'BA.plot' instead.

plot of chunk unnamed-chunk-6

## Limits of agreement:
## Reference lab IU/ml - Home lab IU/ml                           2.5% limit 
##                             -5100327                            -65195645 
##                          97.5% limit                             SD(diff) 
##                             54994992                             30047659

When the dots are around 0, the two test could be interchanged for a patient. There are, however, some outliners: large difference of viral count between the two labs. One difference can be accounted for; the upper limited value of the reference lab is '>170.000.000 IU/ml, whereas the home-lab produces an exact calculation of 288402140 IU/ml.

Wish list for molecular lab

A clinical laboratory in Ethiopia wants to expand on molecular diagnostics. In order to get an idea what the ‘expanses’ are, I prepared a list of consumables, equipment, etc. Actually I was searching Internet if anyone is sharing such a list already, which wasn’t the case. So I prepared my own list which is available for anyone. The prices of all the items are mostly derived from the Fisher Scientific catalogue.

PCR rooms

The molecular lab consists of three physically separated rooms. In low resource settings, air control might be difficult. For PCR-1 it is important that air movement is limited to the minimum. For this, a PCR-cabinet can be used.

Here is the excel file with  the list of items for a molecular diagnostic laboratory. Have fun with it.

checklist PCR labo

In another excel file, I calculated exactly what the costs would be if, for example, the molecular diagnostic lab would perform an X number of multiplex real-time PCRs, for Y number patients, Z times a year. You can ask me if you need help in calculating the investment needed for your lab.

Proposal for a research partnership in Addis Abeba

International Clinical Laboratories. Addis Abebe, Ethiopia.


Prepared by: Dr. Mesfin Nigussie, Medical Director & Dr. Ir. Robert-Jan ten Hove, Parasitologist

Ethiopia – Country profile

Unique among African countries, the ancient Ethiopian monarchy, that maintained its freedom throughout the history. Ethiopia is home to numerous cultures, religions and languages and its total population is estimated at 93,877,025 (July 2013 est.). The climate of Ethiopia is as diverse as the geographical areas. The highest mountain, Ras Dashen rises to an impressive 4620 meters, while the Dalol depression is located 130 meters below sea level. Disease patterns follow this diverse geographical and ethnic diversity. Infectious diseases rank top among public health problems while non-communicable “western” disease are on the rise at an alarming rate.

About International Clinical Laboratories

International Clinical Laboratories (ICL) is a private diagnostic facility, established in Ethiopia in 2004. The mission of ICL is to contribute to the health of the nation while guarantying that the services are both affordable and of high quality. It is the only clinical diagnostic laboratory that is accredited by Joint Commission International (JCI) in sub-Saharan Africa.


ICL main facility is located in the capital, Addis Ababa and houses the main laboratory. The laboratory has over 200 test menus and performs clinical chemistry, hematology, immunoassays, molecular diagnostics, bacteriology, parasitology and mycology. In addition it is in the process of establishing a histopathology / cytopathology section.

The ICL is continuously expanding its operations with patient service centers all over the nation and, in the near future across the East-African region. At the service centers, samples are collected and transported to the main testing facility in Addis Ababa. ICL has unprecedented reach in laboratory services in Ethiopia. This gives us the opportunity to get biological specimens from all over the country and, as such, gives a valuable research opportunity.

Screen Shot 2014-08-01 at 10.54.20 ICL service centers on 1/08/2014

Besides the expansion of testing services, the management of ICL envisions that public health will benefits most by preventive measures. Promoting healthy lifestyle and focus on upcoming healthcare problems will have a more positive impact on the public health than diagnosing and treating the diseases. Scientific research will help understand the healthcare problems that the country has to deal with today, and in the future.

Profile of guest researcher

For the continual improvement of the laboratory services, the ICL is keen on attracting health professionals. We are therefore inviting researchers who are willing to work at the clinical laboratory in Addis Abeba for a period of several weeks or months. The ideal candidate has:

  • A PhD, or is in the trajectory of a PhD in the field of medical microbiology and / or pathology.
  • At least one year of hands-on experience in a routine clinical laboratory
  • Good interpersonal skills to cooperate with local staff and willing to train local personal or supervise local students.
  • Willing to publish and / or present the research to the national and international scientific community.

Examples of research areas

  • Sero-epidemiological studies have shown show that Helicobacter pylori is an important gastric pathogen among the Ethiopian population. There are few data on resistance in Ethiopian strains, complicating the treatment of the associated illnesses. H. pylori tends to be highly localized among families or communities and resistance patterns can be different between strains. Characterizing the H. pylori strains in Addis Abeba should be useful for future management in order to effectively deal with this illness.
  • In several African countries beside Ethiopia, the cause of severe diarrhea in pediatric patients has mainly been attributed to Rotavirus, Shigella, Cryptosporidium and ST-ETEC. In Ethiopia there is limited data on the most important causes of diarrhea both in the pediatric patient group as in the general population. For example, patients with diarrhea are often treated for Entamoeba histolytica based on microscopic examination or for Salmonella without confirmative culture (personal communications). Reliable diagnostic data on infectious diarrheal diseases give insight on the main communicable diarrheal diseases in Addis Abeba. Using molecular diagnostics with subsequent culture and analysis of resistance patterns of the bacterial infections, valuable feedback can be offered to the health-care providers.
  • Ethiopia is one of the fastest growing economies in the world. The lab-work is reflected by the increase of the middle class and changes in lifestyle; all the non-communicable diseases are rising rapidly. The exact impact of diseases such as diabetes, cancer and autoimmune disorders on the Ethiopian public health is largely unknown.
  • What are the most common causes of seasonal flu like symptoms in Ethiopians? Every flu like symptom is ascribed to “colds” and nobody has done any investigation which viruses are the causes, and if it is viral at all.
  • Clinical laboratories in Ethiopia do not have a good functioning quality management system. There is no well organized national proficiency testing (External QA) and therefore nobody knows how reliable or unreliable routine lab tests are in the country. Initiating inter-lab comparison on selected common lab tests will be very interesting for the labs as well as policy makers.
  • Ethiopia is one of the countries in Africa with a high HIV burden. Antiretroviral therapy was initiated in 2005. It has been more than ten years since the treatment program was launched, however, we have no data on drug resistance. Such investigation will be extremely valuable in decision making for treatment and prevention strategies.
  • Cervical cancer is the number one killer in women of reproductive age when infectious causes are disregarded. Because of the seemingly rising incidence of cancers in Ethiopia, the government has started to take non-communicable diseases seriously. There are no national screening programs for cervical cancer and there is limited data on HPV prevalence and types. Factual information is needed to initiate vaccination or screening programs.
  • Molecular characterization of breast carcinoma in Ethiopian patients is an interesting subject. There have been sporadic investigations, however, conclusive molecular characterization is unavailable (ER, PR, HER2, BRAC, etc). With treatment increasingly available, such knowledge will be crucial in medical decision-making.
  • Parasites of the blood and of the GI are common in Ethiopia. Still, we rely on traditional microscopic methods for diagnosis. There seems to be no movement in modernizing this tradition. In the era of molecular diagnosis, can we bring about change to this old method? Is it possible to adapt molecular methods so that we will be able to make molecular parasitological diagnosis affordable and of high quality?


Questions, suggestions and propositions can be directed to:

Dr. Mesfin Nigussie, Medical Director. T +251-11 4671818. E mesfin@icl.com.et

Dr. Robert-Jan ten Hove, Parasitologist. T +251-939881532. E robert@icladdis.com