Skip to contents

Comprehensive content validity analysis

Source: R/content_validity.R
content_validity.Rd

Runs the standard relevance-scale content validity indices on a single ratings matrix and returns a tidy summary. Computes Item-level CVI, modified kappa, Aiken's V, Gwet's AC1, and Gwet's AC2 at the item level; S-CVI/Ave, S-CVI/UA, mean modified kappa, mean AC1, and mean AC2 at the scale level. New AC1 and AC2 columns added in v0.2.0.

Usage

content_validity(
  ratings,
  relevant_threshold = 3,
  lo = 1,
  hi = 4,
  categories = NULL,
  ac2_weights = "quadratic",
  subscale = NULL,
  na.rm = FALSE
)

Arguments

ratings

A numeric matrix or data frame of expert ratings (rows = experts, columns = items) on a relevance scale.

relevant_threshold

Integer. Minimum rating considered "relevant". Passed to icvi(), scvi_ave(), scvi_ua(), mod_kappa(), and gwet_ac1(). Defaults to 3.

lo, hi

Numeric. Minimum and maximum possible rating values on the scale; passed to aiken_v(). Defaults to 1 and 4.

categories

Numeric vector of all possible rating values, used by gwet_ac2(). Defaults to seq(lo, hi), which is correct for the typical 4-point relevance scale.

ac2_weights

Weighting scheme passed to gwet_ac2(). One of "quadratic" (default), "linear", "identity", or a custom square matrix.

subscale

Optional character or factor vector of length ncol(ratings) assigning each item to a subscale (factor / domain). When supplied, the scale-level indices are computed both overall and per-subscale, and the result carries a $subscales data frame. Useful for multi-dimensional instruments where different items measure different constructs. Defaults to NULL (overall only).

na.rm

Logical. Passed to all underlying functions. Defaults to FALSE.

Value

An object of class "content_validity": a list containing

  • items: a data frame with one row per item and columns item, icvi, mod_kappa, aiken_v, gwet_ac1, gwet_ac2.

  • scale: a named numeric vector with scvi_ave, scvi_ua, mean_kappa, mean_ac1, mean_ac2.

  • n_experts: integer, number of experts (rows).

  • n_items: integer, number of items (columns).

Details

Lawshe's CVR is not included in this wrapper because it uses a different rating convention (essential / useful but not essential / not necessary). For CVR analyses, use cvr() and cvr_critical() directly.

See also

icvi(), scvi_ave(), scvi_ua(), mod_kappa(), aiken_v(), gwet_ac1(), gwet_ac2(), cvr()

Examples

ratings <- matrix(
  c(4, 4, 3, 4, 4,
    3, 4, 4, 4, 3,
    2, 3, 3, 4, 3,
    1, 2, 3, 2, 3),
  nrow = 5,
  dimnames = list(NULL, paste0("item", 1:4))
)
result <- content_validity(ratings)
result
#> Content Validity Analysis
#> -------------------------
#> Experts: 5
#> Items:   4
#> 
#> Item-level indices:
#>   item icvi mod_kappa aiken_v gwet_ac1 gwet_ac2
#>  item1  1.0    1.0000  0.9333   1.0000   0.9358
#>  item2  1.0    1.0000  0.8667   1.0000   0.8760
#>  item3  0.8    0.7630  0.6667   0.4118   0.7588
#>  item4  0.4    0.1273  0.4000  -0.1538   0.5946
#> 
#> Scale-level indices (overall):
#>   scvi_ave    scvi_ua mean_kappa   mean_ac1   mean_ac2 
#>     0.8000     0.5000     0.7226     0.5645     0.7913 
result$items
#>    item icvi mod_kappa   aiken_v   gwet_ac1  gwet_ac2
#> 1 item1  1.0 1.0000000 0.9333333  1.0000000 0.9357602
#> 2 item2  1.0 1.0000000 0.8666667  1.0000000 0.8760331
#> 3 item3  0.8 0.7629630 0.6666667  0.4117647 0.7588424
#> 4 item4  0.4 0.1272727 0.4000000 -0.1538462 0.5945946
result$scale
#>   scvi_ave    scvi_ua mean_kappa   mean_ac1   mean_ac2 
#>  0.8000000  0.5000000  0.7225589  0.5644796  0.7913076