Development of chemistry indicators
Download
1 / 78

Development of Chemistry Indicators - PowerPoint PPT Presentation


  • 57 Views
  • Uploaded on

Sediment Quality Objectives For California Enclosed Bays and Estuaries. Development of Chemistry Indicators. Scientific Steering Committee Meeting July 26, 2005. Data screening & processing Strata Calibration & validation subsets. Existing national SQGs Calibration of national SQGs

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Development of Chemistry Indicators' - leila-houston


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Development of chemistry indicators

Sediment Quality Objectives

For California Enclosed Bays and Estuaries

Development of Chemistry Indicators

Scientific Steering Committee Meeting

July 26, 2005


Presentation overview

Data screening & processing

Strata

Calibration & validation subsets

Existing national SQGs

Calibration of national SQGs

New approaches

Categorical classification

Correlation

Predictive ability

Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps


Presentation overview1
Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps


Chemistry indicators
Chemistry Indicators

  • Several challenges to effective use

    • Bioavailability

    • Unmeasured chemicals

    • Mixtures


Objectives
Objectives

  • Identify important geographic, geochemical, or other factors that affect relationship between chemistry and biological effects

  • Develop indicator(s) that reflect relevant biological effects caused by contaminant exposure

  • Develop thresholds and guidance for use in MLOE framework


Approach
Approach

  • Use CA sediment quality data in developing and validating indicators

    • Address concerns and uncertainty regarding influence of regional factors

    • Document performance for realistic applications

  • Investigate multiple approaches

    • Both mechanistic and empirical methods

    • Existing methods used by other programs

    • Existing methods calibrated to California

    • New approaches


Approach1
Approach

  • Evaluate SQG performance

    • Use CA data

    • Use quantitative and consistent approach

    • Select methods with best performance for expected applications

  • Describe response levels (thresholds)

    • Consistent with needs of MLOE framework

    • Based on observed relationships with biological effects


Presentation overview2
Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Data screening & processing

Strata

Calibration & validation subsets


Data screening
Data Screening

  • Appropriate habitat and geographic range

    • Subtidal, embayment, surface sediment samples

  • Chemistry data screening

    • Valid data (from qualifier information)

    • Nondetect values (estimated)

    • Completeness (metals and PAHs)

      • Minimum of 10 chemicals: metals and organics

    • Habitat type (surface, embayment, subtidal)

      Standardized sums:DDTs, PCBs, PAHs, Chlordanes


Data screening1
Data Screening

  • Toxicity data screening

    • Valid data

    • Selection of candidate acute and chronic toxicity test

    • Lack of ammonia interference

      • EPA toxicity test thresholds

    • Acceptable control performance

    • Matched data (toxicity and chemistry)

      • Same station, same sampling event

    • Test method: amphipod mortality only

      • Eohaustorius or Rhepoxynius



Presentation overview3
Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Data screening & processing

Strata

Calibration & validation subsets


Strata
Strata

Are there differences in contamination among regions of CA that are likely to affect the development of a chemical indicator?

  • Geographic Strata

    • North (North of Pt. Conception)

    • South (South of Pt Conception

  • Habitat Strata

    • Ports, Marinas, Shallow

  • Magnitude of contamination

  • Relationship between contamination and toxicity




Strata decisions
Strata Decisions

  • Treat North and South as separate strata

    • Different contamination levels and sources

    • May be different empirical relationships with effects

    • Adequate data for statistical analyses

  • Do not distinguish among habitat regions

    • Limited data for some habitats

    • Added complexity of application


Presentation overview4
Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Data screening & processing

Strata

Calibration & validation subsets


Calibration and validation datasets
Calibration and Validation Datasets

  • Calibration/development dataset

    • Screened data minus withheld validation data

    • Calibration of SQGs

    • Development of new SQGs

    • Comparison of performance

  • Validation dataset

    • Confirm performance of candidate SQGs


Validation dataset
Validation Dataset

  • Independent subset of SQO database plus new studies

  • Approximately 30% of data, selected randomly to represent contamination gradient

  • North and South data are proportional between the calibration/development and validation datasets


Bay estuary samples in database after screening
Bay/Estuary Samples inDatabase After Screening


Presentation overview5
Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Existing national SQGs

Calibration of national SQGs

New approaches


National sqgs
National SQGs

  • Two main types of approaches

    • Empirical and Mechanistic

  • Empirical

    • Intended to aid in prediction of potential for adverse impacts

    • Derived from analysis of extensive field datasets

    • Various approaches for development of chemical values

    • Little explicit consideration of bioavailability

    • Incorporate a wide range of chemicals

    • Work best when applied to mixture of contaminants in a sediment



National sqgs1
National SQGs

  • Mechanistic

    • Intended to assess potential for impacts due to specific chemical groups, not predict overall effects

    • Derived using equilibrium partitioning and toxicological dose-response information

    • Incorporate water quality objectives

    • Explicit consideration of bioavailability

    • Applicable to a restricted range of chemicals

    • Work best when applied to specific contaminants




Presentation overview6
Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Existing national SQGs

Calibration of national SQGs

New approaches


Calibration of national sqgs
Calibration of National SQGs

Objective: Improve empirical relationship between chemistry and effects by modifying national SQGs to address potential sources of uncertainty

  • Variation in bioavailability of organics

  • Variation in natural background concentration of metals

  • CA-Specific variations in chemical mixtures

Differences in organic carbon content of sediment influences exposure

Metal content of sediment matrix varies according to particle type and source material

Relative proportions of contaminants within regions of State may differ from national average


Organics bioavailability calibration
Organics Bioavailability Calibration

  • TOC normalization to represent changes in bioavailability

    • Conc./TOC

  • Evaluate whether predictive relationship for chemical classes is improved after normalization

    • Correlation analysis

  • Use normalized values as basis for SQG calibration if there is evidence of improved predictive relationship


Toc normalization
TOC Normalization

Relationship to sediment toxicity is not improved by TOC normalization of organics


Metal background calibration
Metal Background Calibration

  • Metals occur naturally in the environment

    • Silts and clays have higher metal content

    • Source of uncertainty in identifying anthropogenic impact

    • Background varies due to sediment type and regional differences in geology

  • Need to differentiate between natural background levels and anthropogenic input

    • Investigate utility for empirical guideline development

    • Potential use for establishing regional background levels


Reference element normalization
Reference Element Normalization

  • Established methodology applied by geologists and environmental scientists

  • Reference element covaries with natural sediment metals and is insensitive to anthropogenic inputs

    • Regression between reference element and metal developed using a dataset of uncontaminated samples

    • Regression line indicates natural background metal concentration for different sediment particle size composition

  • Use of iron as reference element validated for southern California

    • 1994 and 1998 Bight regional surveys


Iron normalization approach
Iron Normalization Approach

  • Log transformed data

  • Selected subset of “reference” stations from SQO database

    • Least potential for anthropogenic metal enrichment

    • Nontoxic stations in lowest 30th percentile of DDT, PCB, and PAH concentrations

    • Reviewed selected stations using GIS to eliminate redundant and likely impacted sites

  • Calculated regressions

  • Used residuals from regression as normalized values

    • Compared relationship of normalized/non -normalized data to toxicity


Southern california results

Zinc

Iron (%)

Southern California Results

Significant regressions obtained for metals of interests in all strata


Residual calculation

Zinc

Residual = actual-predicted concentration

Iron (%)

Residual Calculation

Residual = relative metal enrichment

Used for correlation analysis with amphipod mortality


Iron normalization
Iron Normalization

Relationship to sediment toxicity is not improved by iron normalization of metals


Normalization summary
Normalization Summary

  • TOC and iron normalization are apparently not effective for improving relationships between chemistry and toxicity

  • Have not pursued use of normalized data in calibrating/developing SQGs

  • Iron normalization may be useful for establishing background metal levels


Calibration of sqgs
Calibration of SQGs

  • Adjustment of models or chemical specific values based on California data

  • Logistic Regression Model (Pmax)

    • Excluded individual chemical models with poor fit

      • Antimony, Arsenic, Chromium, Nickel

    • Adjusted Pmax model to fit CA data (N, S, All)

  • ERM

    • Derived CA-specific values using modified method of Ingersoll et al.

    • Sample-based analysis


Ca erm calculation
CA ERM Calculation

  • Select paired chemistry and amphipod toxicity data by stratum

    • Log transform all chemistry data

  • Classify samples as toxic/nontoxic based on 20% mortality threshold

  • Calculate median concentration of the nontoxic samples

  • Select only those toxic samples where concentration of individual chemicals > 2x nontoxic median

  • CA ERM = median concentration from screened toxic samples

    • At least 10 toxic samples required for ERM calculation


Substantial differences in some ERM values derived for California datasets compared to nationally derived values


Presentation overview7
Presentation Overview California datasets compared to nationally derived values

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Existing national SQGs

Calibration of national SQGs

New approaches


New sqg characteristics
New SQG Characteristics California datasets compared to nationally derived values

  • Categorical classification and multiple thresholds

  • Based on individual chemical models or values

  • Thresholds can be adjusted

  • Accept continuous and categorical data

  • Some type of weighting based on strength of relationship

  • Compatible with multiple line of evidence assessment framework

  • Capability to include/adapt to new contaminants of concern

  • Adaptable to different application objectives

  • Able to use toxicity and benthic community impact data in development

  • Result reflects uncertainty of empirical relationship


Kappa statistic
Kappa Statistic California datasets compared to nationally derived values

  • Developed in 1960-70’s

    • Peer-reviewed literature describes derivation and interpretation

  • Used in medicine, epidemiology, & psychology to evaluate observer agreement/reliability

    • Similar problem to SQG development and assessment

    • Accommodates multiple categories of classification

    • Multiple thresholds can be adjusted by user

    • Categorical or ordinal data

    • Result reflects magnitude of disagreement (can be used to weight values)

  • Sediment quality assessment is a new application


Kappa

Toxicity Result California datasets compared to nationally derived values

SQG Result

High

Moderate

Marginal

Reference

High

Moderate

Low

Reference

T3

T2

T1

Kappa

Evaluates agreement between 2 methods of classification

  • Chemical SQG result

  • Toxicity test result

  • Magnitude of error affects score


Toxicity California datasets compared to nationally derived values

Kappa = 0.48 

SQG Category

High

Moderate

Marginal

Reference

High

60

30

20

1

Moderate

33

50

25

0

Low

10

14

65

6

Reference

3

7

20

25

Chemical 1Good Association Between Concentration and Effect(most of errors in cells adjacent to diagonal)


Toxicity California datasets compared to nationally derived values

Kappa = 0.27 

SQG Category

High

Moderate

Marginal

Reference

High

60

1

20

30

Moderate

33

50

0

25

Low

14

10

65

6

Reference

20

7

3

25

Chemical 2 Poor Association Between Concentration and Effect(more errors in categories distant from diagonal)


Kappa analysis output
Kappa Analysis Output California datasets compared to nationally derived values

  • Kappa (k)

    • Similar to correlation coefficient

    • Confidence intervals

  • Multiple thresholds

    • Optimized for correspondence to effect levels

    • Applied to other data to predict effect category (cat)

      • E.g., Category 1, 2, 3, or 4


New kappa sqgs
New Kappa SQGs California datasets compared to nationally derived values

  • Derived Kappa and thresholds for target chemicals using amphipod mortality data

    • As, Cd, Cr, Cu, Pb, Hg, Ni, Ag, Zn , t chlordane, t DDT, t PAH, t PCB

  • Calculated Kappa score for each chemical in sample

    • k x cat

  • Mean weighted Kappa score

    • Average of k x cat

    • Each constituent contributes to final classification in a manner proportional to reliability of relationship

    • Mixture joint effects model

  • Maximum Kappa

    • Highest Kappa score for any individual chemical

    • Independent mixture effects model


Presentation overview8
Presentation Overview California datasets compared to nationally derived values

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Categorical classification

Correlation

Predictive ability


Evaluation process
Evaluation Process California datasets compared to nationally derived values

  • Compare performance of candidate SQG approaches in a manner relevant to desired application

    • Ability to accurately classify presence and magnitude of biological effects based on chemistry

    • California marine embayment data

  • Use statistical measures to identify short list of best performing approaches

    • Categorical classification

    • Correlation

  • Validate performance results

    • Validation dataset

  • Rank candidate approaches

  • Examine significance of differences

    • Predictive ability


Evaluation of sqgs
Evaluation of SQGs California datasets compared to nationally derived values

  • Categorical (ability to classify each station into one of four toxicity response categories)

    • Kappa value

    • Level 1=<10% mortality, Level 2=10-20%, Level 3=20-40%, Level 4=>40%

    • SQG thresholds optimized for best score

  • Spearman’s correlation coefficient

    • Nonparametric measure of association

    • Independent of Kappa calculation

  • Validation

    • Used same thresholds selected for calibration dataset


Sqg evaluation north

Spearman California datasets compared to nationally derived values

Correlation

SQG

Kappa

Mean Weighted Kappa

0.54

0.36

Northern CA mERMq

0.37

0.29

Northern CA Pmax

0.35

0.29

Max Weighted Kappa

0.40

0.26

mConsensusq

0.29

0.24

mERMq

0.37

0.24

mSQGQ1q

0.28

0.22

National Pmax

0.27

0.19

Chronic EqP TU

-0.08

0.08

Acute EqP TU

-0.09

0.08

SQG Evaluation: North


Sqg evaluation south

Spearman California datasets compared to nationally derived values

Correlation

SQG

Kappa

Mean Weighted Kappa

0.46

0.31

Max Weighted Kappa

0.43

0.27

Southern CA Pmax

0.32

0.21

mERMq

0.29

0.18

Southern CA mERMq

0.28

0.18

National Pmax

0.22

0.16

mSQGQ1q

0.25

0.16

mConsensusq

0.22

0.13

Chronic EqP TU

-0.06

0.04

Acute EqP TU

-0.08

0.03

SQG Evaluation:South


Sqg validation north

Spearman California datasets compared to nationally derived values

Correlation

SQG

Kappa

Mean Weighted Kappa

0.47

0.31

Northern CA mERMq

0.38

0.26

Max Weighted Kappa

0.36

0.22

Northern CA Pmax

0.31

0.21

mSQGQ1q

0.38

0.21

National Pmax

0.30

0.17

mERMq

0.31

0.17

mConsensusq

0.14

0.12

SQG Validation:North

All top ranked SQGs validate


Sqg validation south

Spearman California datasets compared to nationally derived values

Correlation

SQG

Kappa

National Pmax

0.34

0.24

Southern CA Pmax

0.34

0.21

mSQGQ1q

0.39

0.21

Mean Weighted Kappa

0.36

0.20

mERMq

0.29

0.18

Max Weighted Kappa

0.34

0.18

Southern CA mERMq

0.22

0.15

mConsensusq

0.21

0.07

SQG Validation:South

All top ranked SQGs validate


Significance of differences
Significance of Differences California datasets compared to nationally derived values

  • Are the differences in performance significant to the user?

    • Do differences in SQG ranking correspond to greater accuracy, applicability, or utility of the SQG?

    • Better predictive ability (efficiency)?

    • Better sensitivity or specificity?

  • Need to look at the data


Sqgs applied to so ca data

NOAA ERM California datasets compared to nationally derived values

Mean Weighted Kappa

So CA ERM

EqP Acute

SQGs Applied to So CA Data


Predictive ability

C California datasets compared to nationally derived values

D

B

A

Predictive Ability

Negative Predictive Value=C/(C+A) x 100(percent of no hits that are nontoxic)=Nontoxic Efficiency

Specificity=C/(C+D) x 100(percent of all nontoxic samples that are classified as a no hit)

Positive Predictive Value=B/(B+D) x 100(percent of hits that are toxic)=Toxic Efficiency

Sensitivity=B/(B+A) x 100(percent of all toxic samples that are classified as a hit)


South mermq
South: mERMq California datasets compared to nationally derived values

  • SQG performance is threshold dependent

  • Inverse relationship between efficiency (toxic or nontoxic) and specificity or sensitivity

  • Improved SQG accuracy when greater efficiency obtained

  • Improved SQG utility when greater sensitivity or specificity obtained without sacrificing efficiency


South mermq1
South: mERMq California datasets compared to nationally derived values

  • Plots of efficiency vs. specificity or sensitivity illustrate tradeoffs in SQG performance at different thresholds


South candidate sqgs
South: Candidate SQGs California datasets compared to nationally derived values

  • Mean weighted Kappa shows improved overall utility for distinguishing both nontoxic and toxic samples


North candidate sqgs
North: Candidate SQGs California datasets compared to nationally derived values

  • Mean weighted Kappa shows improved specificity and toxic efficiency


Evaluation and validation summary
Evaluation and Validation Summary California datasets compared to nationally derived values

  • North

    • Mean weighted Kappa has highest performance

    • Northern California ERM and Northern California Pmax also perform better than others

  • South

    • Mean weighted Kappa has highest performance

    • Max Kappa also performs better than others

  • Validation results consistent with evaluation

    • The approaches are robust


Presentation overview9
Presentation Overview California datasets compared to nationally derived values

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps


Conclusions
Conclusions California datasets compared to nationally derived values

  • Pursue mean weighted Kappa as component of chemistry LOE

    • Best relationship with toxicity

    • Easily adaptable to new chemicals or different datasets

    • Provides information on strength of relationship

  • Use EqP benchmarks as component of stressor identification, not chemical LOE score

    • Predictive value not strong enough

    • Provide guidance on calculation and interpretation


Presentation overview10
Presentation Overview California datasets compared to nationally derived values

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Thresholds

Benthos


Options for threshold development
Options for Threshold Development California datasets compared to nationally derived values

  • Optimum statistical fit to effects in CA

    • Toxicity only?

    • Benthos only?

    • Combination?

  • Based on accuracy or error rate

  • Consideration of national patterns


National vs ca data
National vs. CA data California datasets compared to nationally derived values

North

South

  • Narrower contamination range in CA

  • High range threshold (1.5) of limited utility


Benthos
Benthos California datasets compared to nationally derived values

  • How should benthic community response be incorporated into the chemical LOE

    • In the SQG approach?

    • In the thresholds?

  • Factors to consider:

    • Strength of relationship between benthos and chemistry or toxicity

    • Relative sensitivity of benthos and toxicity responses

    • Nature of association with chemistry

    • Are there different drivers?


Benthos1
Benthos California datasets compared to nationally derived values

Preliminary data analysis:

  • Used existing benthic response index (BRI) data for So. Calif. and San Francisco Bay

    • South San Francisco Bay (North); n=83

    • Southern California (South); n=203

  • Examined three aspects of relationship with chemistry

    • Strength of relationship with SQGs and chemicals

    • Relative sensitivity of response compared to toxicity

    • Chemical drivers


Benthos2
Benthos California datasets compared to nationally derived values


Benthos3
Benthos California datasets compared to nationally derived values

  • Significant correlations are present between BRI scores and SQGs or individual chemicals


Benthos4
Benthos California datasets compared to nationally derived values

  • Strong correlation between benthic response and amphipod mortality

  • Benthic response when no toxicity is evident


Relative sensitivity of benthos response

Proportion of Samples California datasets compared to nationally derived values

Proportion of Samples

Relative Sensitivity of Benthos Response

  • Use cumulative distribution function to indicate approximate thresholds for increased incidence of impacts (10th percentile) and likely impacts (50th percentile)

  • Compare results for toxicity and benthos (same dataset)


Relative sensitivity of benthos response1
Relative Sensitivity of Benthos Response California datasets compared to nationally derived values

  • Toxicity and benthos responses occur over similar contamination ranges


Chemical correlations north

S. Correlation California datasets compared to nationally derived values

S. Correlation

Chemical Correlations : North

Benthos

Chlordane, copper, and zinc show different relative influence on effects

Toxicity


Chemical correlations south

S. Correlation California datasets compared to nationally derived values

S. Correlation

Chemical Correlations: South

Benthos

Cadmium, DDTs, and zinc show different relative influence on effects

Toxicity


Recommendations
Recommendations California datasets compared to nationally derived values

  • Develop thresholds of application specific to toxicity and benthos

    • Need to incorporate both types of responses into assessment

  • Continue development of a SQG that is best predictor of benthic community impacts

    • May respond to different chemical mixtures

    • Need revised benthic index data to complete development and evaluation

    • Determine whether toxicity and benthos SQGs are needed

    • A method to combine the results will be needed to produce a single chemistry LOE score


ad