Development of chemistry indicators
This presentation is the property of its rightful owner.
Sponsored Links
1 / 78

Development of Chemistry Indicators PowerPoint PPT Presentation


  • 37 Views
  • Uploaded on
  • Presentation posted in: General

Sediment Quality Objectives For California Enclosed Bays and Estuaries. Development of Chemistry Indicators. Scientific Steering Committee Meeting July 26, 2005. Data screening & processing Strata Calibration & validation subsets. Existing national SQGs Calibration of national SQGs

Download Presentation

Development of Chemistry Indicators

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Development of chemistry indicators

Sediment Quality Objectives

For California Enclosed Bays and Estuaries

Development of Chemistry Indicators

Scientific Steering Committee Meeting

July 26, 2005


Presentation overview

Data screening & processing

Strata

Calibration & validation subsets

Existing national SQGs

Calibration of national SQGs

New approaches

Categorical classification

Correlation

Predictive ability

Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps


Presentation overview1

Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps


Chemistry indicators

Chemistry Indicators

  • Several challenges to effective use

    • Bioavailability

    • Unmeasured chemicals

    • Mixtures


Objectives

Objectives

  • Identify important geographic, geochemical, or other factors that affect relationship between chemistry and biological effects

  • Develop indicator(s) that reflect relevant biological effects caused by contaminant exposure

  • Develop thresholds and guidance for use in MLOE framework


Approach

Approach

  • Use CA sediment quality data in developing and validating indicators

    • Address concerns and uncertainty regarding influence of regional factors

    • Document performance for realistic applications

  • Investigate multiple approaches

    • Both mechanistic and empirical methods

    • Existing methods used by other programs

    • Existing methods calibrated to California

    • New approaches


Approach1

Approach

  • Evaluate SQG performance

    • Use CA data

    • Use quantitative and consistent approach

    • Select methods with best performance for expected applications

  • Describe response levels (thresholds)

    • Consistent with needs of MLOE framework

    • Based on observed relationships with biological effects


Presentation overview2

Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Data screening & processing

Strata

Calibration & validation subsets


Data screening

Data Screening

  • Appropriate habitat and geographic range

    • Subtidal, embayment, surface sediment samples

  • Chemistry data screening

    • Valid data (from qualifier information)

    • Nondetect values (estimated)

    • Completeness (metals and PAHs)

      • Minimum of 10 chemicals: metals and organics

    • Habitat type (surface, embayment, subtidal)

      Standardized sums:DDTs, PCBs, PAHs, Chlordanes


Data screening1

Data Screening

  • Toxicity data screening

    • Valid data

    • Selection of candidate acute and chronic toxicity test

    • Lack of ammonia interference

      • EPA toxicity test thresholds

    • Acceptable control performance

    • Matched data (toxicity and chemistry)

      • Same station, same sampling event

    • Test method: amphipod mortality only

      • Eohaustorius or Rhepoxynius


Data screening2

Data Screening


Presentation overview3

Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Data screening & processing

Strata

Calibration & validation subsets


Strata

Strata

Are there differences in contamination among regions of CA that are likely to affect the development of a chemical indicator?

  • Geographic Strata

    • North (North of Pt. Conception)

    • South (South of Pt Conception

  • Habitat Strata

    • Ports, Marinas, Shallow

  • Magnitude of contamination

  • Relationship between contamination and toxicity


Strata1

Strata


Strata2

Strata


Strata decisions

Strata Decisions

  • Treat North and South as separate strata

    • Different contamination levels and sources

    • May be different empirical relationships with effects

    • Adequate data for statistical analyses

  • Do not distinguish among habitat regions

    • Limited data for some habitats

    • Added complexity of application


Presentation overview4

Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Data screening & processing

Strata

Calibration & validation subsets


Calibration and validation datasets

Calibration and Validation Datasets

  • Calibration/development dataset

    • Screened data minus withheld validation data

    • Calibration of SQGs

    • Development of new SQGs

    • Comparison of performance

  • Validation dataset

    • Confirm performance of candidate SQGs


Validation dataset

Validation Dataset

  • Independent subset of SQO database plus new studies

  • Approximately 30% of data, selected randomly to represent contamination gradient

  • North and South data are proportional between the calibration/development and validation datasets


Bay estuary samples in database after screening

Bay/Estuary Samples inDatabase After Screening


Presentation overview5

Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Existing national SQGs

Calibration of national SQGs

New approaches


National sqgs

National SQGs

  • Two main types of approaches

    • Empirical and Mechanistic

  • Empirical

    • Intended to aid in prediction of potential for adverse impacts

    • Derived from analysis of extensive field datasets

    • Various approaches for development of chemical values

    • Little explicit consideration of bioavailability

    • Incorporate a wide range of chemicals

    • Work best when applied to mixture of contaminants in a sediment


Empirical sqgs

Empirical SQGs


National sqgs1

National SQGs

  • Mechanistic

    • Intended to assess potential for impacts due to specific chemical groups, not predict overall effects

    • Derived using equilibrium partitioning and toxicological dose-response information

    • Incorporate water quality objectives

    • Explicit consideration of bioavailability

    • Applicable to a restricted range of chemicals

    • Work best when applied to specific contaminants


Mechanistic sqgs

Mechanistic SQGs


National sqgs2

National SQGs


Presentation overview6

Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Existing national SQGs

Calibration of national SQGs

New approaches


Calibration of national sqgs

Calibration of National SQGs

Objective: Improve empirical relationship between chemistry and effects by modifying national SQGs to address potential sources of uncertainty

  • Variation in bioavailability of organics

  • Variation in natural background concentration of metals

  • CA-Specific variations in chemical mixtures

Differences in organic carbon content of sediment influences exposure

Metal content of sediment matrix varies according to particle type and source material

Relative proportions of contaminants within regions of State may differ from national average


Organics bioavailability calibration

Organics Bioavailability Calibration

  • TOC normalization to represent changes in bioavailability

    • Conc./TOC

  • Evaluate whether predictive relationship for chemical classes is improved after normalization

    • Correlation analysis

  • Use normalized values as basis for SQG calibration if there is evidence of improved predictive relationship


Toc normalization

TOC Normalization

Relationship to sediment toxicity is not improved by TOC normalization of organics


Metal background calibration

Metal Background Calibration

  • Metals occur naturally in the environment

    • Silts and clays have higher metal content

    • Source of uncertainty in identifying anthropogenic impact

    • Background varies due to sediment type and regional differences in geology

  • Need to differentiate between natural background levels and anthropogenic input

    • Investigate utility for empirical guideline development

    • Potential use for establishing regional background levels


Reference element normalization

Reference Element Normalization

  • Established methodology applied by geologists and environmental scientists

  • Reference element covaries with natural sediment metals and is insensitive to anthropogenic inputs

    • Regression between reference element and metal developed using a dataset of uncontaminated samples

    • Regression line indicates natural background metal concentration for different sediment particle size composition

  • Use of iron as reference element validated for southern California

    • 1994 and 1998 Bight regional surveys


Iron normalization approach

Iron Normalization Approach

  • Log transformed data

  • Selected subset of “reference” stations from SQO database

    • Least potential for anthropogenic metal enrichment

    • Nontoxic stations in lowest 30th percentile of DDT, PCB, and PAH concentrations

    • Reviewed selected stations using GIS to eliminate redundant and likely impacted sites

  • Calculated regressions

  • Used residuals from regression as normalized values

    • Compared relationship of normalized/non -normalized data to toxicity


Southern california results

Zinc

Iron (%)

Southern California Results

Significant regressions obtained for metals of interests in all strata


Residual calculation

Zinc

Residual = actual-predicted concentration

Iron (%)

Residual Calculation

Residual = relative metal enrichment

Used for correlation analysis with amphipod mortality


Iron normalization

Iron Normalization

Relationship to sediment toxicity is not improved by iron normalization of metals


Normalization summary

Normalization Summary

  • TOC and iron normalization are apparently not effective for improving relationships between chemistry and toxicity

  • Have not pursued use of normalized data in calibrating/developing SQGs

  • Iron normalization may be useful for establishing background metal levels


Calibration of sqgs

Calibration of SQGs

  • Adjustment of models or chemical specific values based on California data

  • Logistic Regression Model (Pmax)

    • Excluded individual chemical models with poor fit

      • Antimony, Arsenic, Chromium, Nickel

    • Adjusted Pmax model to fit CA data (N, S, All)

  • ERM

    • Derived CA-specific values using modified method of Ingersoll et al.

    • Sample-based analysis


Ca erm calculation

CA ERM Calculation

  • Select paired chemistry and amphipod toxicity data by stratum

    • Log transform all chemistry data

  • Classify samples as toxic/nontoxic based on 20% mortality threshold

  • Calculate median concentration of the nontoxic samples

  • Select only those toxic samples where concentration of individual chemicals > 2x nontoxic median

  • CA ERM = median concentration from screened toxic samples

    • At least 10 toxic samples required for ERM calculation


Development of chemistry indicators

Substantial differences in some ERM values derived for California datasets compared to nationally derived values


Presentation overview7

Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Existing national SQGs

Calibration of national SQGs

New approaches


New sqg characteristics

New SQG Characteristics

  • Categorical classification and multiple thresholds

  • Based on individual chemical models or values

  • Thresholds can be adjusted

  • Accept continuous and categorical data

  • Some type of weighting based on strength of relationship

  • Compatible with multiple line of evidence assessment framework

  • Capability to include/adapt to new contaminants of concern

  • Adaptable to different application objectives

  • Able to use toxicity and benthic community impact data in development

  • Result reflects uncertainty of empirical relationship


Kappa statistic

Kappa Statistic

  • Developed in 1960-70’s

    • Peer-reviewed literature describes derivation and interpretation

  • Used in medicine, epidemiology, & psychology to evaluate observer agreement/reliability

    • Similar problem to SQG development and assessment

    • Accommodates multiple categories of classification

    • Multiple thresholds can be adjusted by user

    • Categorical or ordinal data

    • Result reflects magnitude of disagreement (can be used to weight values)

  • Sediment quality assessment is a new application


Kappa

Toxicity Result

SQG Result

High

Moderate

Marginal

Reference

High

Moderate

Low

Reference

T3

T2

T1

Kappa

Evaluates agreement between 2 methods of classification

  • Chemical SQG result

  • Toxicity test result

  • Magnitude of error affects score


Development of chemistry indicators

Toxicity

Kappa = 0.48 

SQG Category

High

Moderate

Marginal

Reference

High

60

30

20

1

Moderate

33

50

25

0

Low

10

14

65

6

Reference

3

7

20

25

Chemical 1Good Association Between Concentration and Effect(most of errors in cells adjacent to diagonal)


Development of chemistry indicators

Toxicity

Kappa = 0.27 

SQG Category

High

Moderate

Marginal

Reference

High

60

1

20

30

Moderate

33

50

0

25

Low

14

10

65

6

Reference

20

7

3

25

Chemical 2 Poor Association Between Concentration and Effect(more errors in categories distant from diagonal)


Kappa analysis output

Kappa Analysis Output

  • Kappa (k)

    • Similar to correlation coefficient

    • Confidence intervals

  • Multiple thresholds

    • Optimized for correspondence to effect levels

    • Applied to other data to predict effect category (cat)

      • E.g., Category 1, 2, 3, or 4


New kappa sqgs

New Kappa SQGs

  • Derived Kappa and thresholds for target chemicals using amphipod mortality data

    • As, Cd, Cr, Cu, Pb, Hg, Ni, Ag, Zn , t chlordane, t DDT, t PAH, t PCB

  • Calculated Kappa score for each chemical in sample

    • k x cat

  • Mean weighted Kappa score

    • Average of k x cat

    • Each constituent contributes to final classification in a manner proportional to reliability of relationship

    • Mixture joint effects model

  • Maximum Kappa

    • Highest Kappa score for any individual chemical

    • Independent mixture effects model


Presentation overview8

Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Categorical classification

Correlation

Predictive ability


Evaluation process

Evaluation Process

  • Compare performance of candidate SQG approaches in a manner relevant to desired application

    • Ability to accurately classify presence and magnitude of biological effects based on chemistry

    • California marine embayment data

  • Use statistical measures to identify short list of best performing approaches

    • Categorical classification

    • Correlation

  • Validate performance results

    • Validation dataset

  • Rank candidate approaches

  • Examine significance of differences

    • Predictive ability


Evaluation of sqgs

Evaluation of SQGs

  • Categorical (ability to classify each station into one of four toxicity response categories)

    • Kappa value

    • Level 1=<10% mortality, Level 2=10-20%, Level 3=20-40%, Level 4=>40%

    • SQG thresholds optimized for best score

  • Spearman’s correlation coefficient

    • Nonparametric measure of association

    • Independent of Kappa calculation

  • Validation

    • Used same thresholds selected for calibration dataset


Sqg evaluation north

Spearman

Correlation

SQG

Kappa

Mean Weighted Kappa

0.54

0.36

Northern CA mERMq

0.37

0.29

Northern CA Pmax

0.35

0.29

Max Weighted Kappa

0.40

0.26

mConsensusq

0.29

0.24

mERMq

0.37

0.24

mSQGQ1q

0.28

0.22

National Pmax

0.27

0.19

Chronic EqP TU

-0.08

0.08

Acute EqP TU

-0.09

0.08

SQG Evaluation: North


Sqg evaluation south

Spearman

Correlation

SQG

Kappa

Mean Weighted Kappa

0.46

0.31

Max Weighted Kappa

0.43

0.27

Southern CA Pmax

0.32

0.21

mERMq

0.29

0.18

Southern CA mERMq

0.28

0.18

National Pmax

0.22

0.16

mSQGQ1q

0.25

0.16

mConsensusq

0.22

0.13

Chronic EqP TU

-0.06

0.04

Acute EqP TU

-0.08

0.03

SQG Evaluation:South


Sqg validation north

Spearman

Correlation

SQG

Kappa

Mean Weighted Kappa

0.47

0.31

Northern CA mERMq

0.38

0.26

Max Weighted Kappa

0.36

0.22

Northern CA Pmax

0.31

0.21

mSQGQ1q

0.38

0.21

National Pmax

0.30

0.17

mERMq

0.31

0.17

mConsensusq

0.14

0.12

SQG Validation:North

All top ranked SQGs validate


Sqg validation south

Spearman

Correlation

SQG

Kappa

National Pmax

0.34

0.24

Southern CA Pmax

0.34

0.21

mSQGQ1q

0.39

0.21

Mean Weighted Kappa

0.36

0.20

mERMq

0.29

0.18

Max Weighted Kappa

0.34

0.18

Southern CA mERMq

0.22

0.15

mConsensusq

0.21

0.07

SQG Validation:South

All top ranked SQGs validate


Significance of differences

Significance of Differences

  • Are the differences in performance significant to the user?

    • Do differences in SQG ranking correspond to greater accuracy, applicability, or utility of the SQG?

    • Better predictive ability (efficiency)?

    • Better sensitivity or specificity?

  • Need to look at the data


Sqgs applied to so ca data

NOAA ERM

Mean Weighted Kappa

So CA ERM

EqP Acute

SQGs Applied to So CA Data


Predictive ability

C

D

B

A

Predictive Ability

Negative Predictive Value=C/(C+A) x 100(percent of no hits that are nontoxic)=Nontoxic Efficiency

Specificity=C/(C+D) x 100(percent of all nontoxic samples that are classified as a no hit)

Positive Predictive Value=B/(B+D) x 100(percent of hits that are toxic)=Toxic Efficiency

Sensitivity=B/(B+A) x 100(percent of all toxic samples that are classified as a hit)


South mermq

South: mERMq

  • SQG performance is threshold dependent

  • Inverse relationship between efficiency (toxic or nontoxic) and specificity or sensitivity

  • Improved SQG accuracy when greater efficiency obtained

  • Improved SQG utility when greater sensitivity or specificity obtained without sacrificing efficiency


South mermq1

South: mERMq

  • Plots of efficiency vs. specificity or sensitivity illustrate tradeoffs in SQG performance at different thresholds


South candidate sqgs

South: Candidate SQGs

  • Mean weighted Kappa shows improved overall utility for distinguishing both nontoxic and toxic samples


North candidate sqgs

North: Candidate SQGs

  • Mean weighted Kappa shows improved specificity and toxic efficiency


Evaluation and validation summary

Evaluation and Validation Summary

  • North

    • Mean weighted Kappa has highest performance

    • Northern California ERM and Northern California Pmax also perform better than others

  • South

    • Mean weighted Kappa has highest performance

    • Max Kappa also performs better than others

  • Validation results consistent with evaluation

    • The approaches are robust


Presentation overview9

Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps


Conclusions

Conclusions

  • Pursue mean weighted Kappa as component of chemistry LOE

    • Best relationship with toxicity

    • Easily adaptable to new chemicals or different datasets

    • Provides information on strength of relationship

  • Use EqP benchmarks as component of stressor identification, not chemical LOE score

    • Predictive value not strong enough

    • Provide guidance on calculation and interpretation


Presentation overview10

Presentation Overview

  • Objectives

  • Data preparation

  • SQG calibration and development

  • Validation

  • Conclusions

  • Next steps

Thresholds

Benthos


Options for threshold development

Options for Threshold Development

  • Optimum statistical fit to effects in CA

    • Toxicity only?

    • Benthos only?

    • Combination?

  • Based on accuracy or error rate

  • Consideration of national patterns


National vs ca data

National vs. CA data

North

South

  • Narrower contamination range in CA

  • High range threshold (1.5) of limited utility


Benthos

Benthos

  • How should benthic community response be incorporated into the chemical LOE

    • In the SQG approach?

    • In the thresholds?

  • Factors to consider:

    • Strength of relationship between benthos and chemistry or toxicity

    • Relative sensitivity of benthos and toxicity responses

    • Nature of association with chemistry

    • Are there different drivers?


Benthos1

Benthos

Preliminary data analysis:

  • Used existing benthic response index (BRI) data for So. Calif. and San Francisco Bay

    • South San Francisco Bay (North); n=83

    • Southern California (South); n=203

  • Examined three aspects of relationship with chemistry

    • Strength of relationship with SQGs and chemicals

    • Relative sensitivity of response compared to toxicity

    • Chemical drivers


Benthos2

Benthos


Benthos3

Benthos

  • Significant correlations are present between BRI scores and SQGs or individual chemicals


Benthos4

Benthos

  • Strong correlation between benthic response and amphipod mortality

  • Benthic response when no toxicity is evident


Relative sensitivity of benthos response

Proportion of Samples

Proportion of Samples

Relative Sensitivity of Benthos Response

  • Use cumulative distribution function to indicate approximate thresholds for increased incidence of impacts (10th percentile) and likely impacts (50th percentile)

  • Compare results for toxicity and benthos (same dataset)


Relative sensitivity of benthos response1

Relative Sensitivity of Benthos Response

  • Toxicity and benthos responses occur over similar contamination ranges


Chemical correlations north

S. Correlation

S. Correlation

Chemical Correlations : North

Benthos

Chlordane, copper, and zinc show different relative influence on effects

Toxicity


Chemical correlations south

S. Correlation

S. Correlation

Chemical Correlations: South

Benthos

Cadmium, DDTs, and zinc show different relative influence on effects

Toxicity


Recommendations

Recommendations

  • Develop thresholds of application specific to toxicity and benthos

    • Need to incorporate both types of responses into assessment

  • Continue development of a SQG that is best predictor of benthic community impacts

    • May respond to different chemical mixtures

    • Need revised benthic index data to complete development and evaluation

    • Determine whether toxicity and benthos SQGs are needed

    • A method to combine the results will be needed to produce a single chemistry LOE score


  • Login