Stan highlighting needs fixing

Information

• Language: Stan

Description Lots of updates to Stan since 2.24 plus a few other things.

First the other things. I believe the Stan code needs an additional parameter to indicate the “blocks” of the language:

• Code blocks should have a color dedicated to them
  • functions, data, transformed data, parameters, transformed parameters, model, generated quantities

Changes since 2.28

• array[] is now a valid keyword
• complex is a valid keyword literal

The higher order functions that are valid are:

      ## Algebraic equation solver
      "algebra_solver", "algebra_solver_newton",

      ## Ordinary differential equation
      "ode_rk45", "ode_rk45_tol", "ode_ckrk", "ode_ckrk_tol", "ode_adams",
      "ode_adams_tol", "ode_bdf", "ode_bdf_tol", "ode_adjoint_tol_ctl",

      ## 1D integrator
      "integrate_1d",

      ## Reduce-sum function
      "reduce_sum", "reduce_sum_static",

It’s probably easier to just have all the functions. I’ve grabbed all this from a recent pr request in the rouge library

Long list of functions

# Integer-Valued Basic Functions

      ## Absolute functions
      "abs", "int_step",

      ## Bound functions
      "min", "max",

      ## Size functions
      "size",

      # Real-Valued Basic Functions

      ## Log probability function
      "target", "get_lp",

      ## Logical functions
      "step", "is_inf", "is_nan",

      ## Step-like functions
      "fabs", "fdim", "fmin", "fmax", "fmod", "floor", "ceil", "round",
      "trunc",

      ## Power and logarithm functions
      "sqrt", "cbrt", "square", "exp", "exp2", "log", "log2", "log10",
      "pow", "inv", "inv_sqrt", "inv_square",

      ## Trigonometric functions
      "hypot", "cos", "sin", "tan", "acos", "asin", "atan", "atan2",

      ## Hyperbolic trigonometric functions
      "cosh", "sinh", "tanh", "acosh", "asinh", "atanh",

      ## Link functions
      "logit", "inv_logit", "inv_cloglog",

      ## Probability-related functions
      "erf", "erfc", "Phi", "inv_Phi", "Phi_approx", "binary_log_loss",
      "owens_t",

      ## Combinatorial functions
      "beta", "inc_beta", "lbeta", "tgamma", "lgamma", "digamma",
      "trigamma", "lmgamma", "gamma_p", "gamma_q",
      "binomial_coefficient_log", "choose", "bessel_first_kind",
      "bessel_second_kind", "modified_bessel_first_kind",
      "log_modified_bessel_first_kind", "modified_bessel_second_kind",
      "falling_factorial", "lchoose", "log_falling_factorial",
      "rising_factorial", "log_rising_factorial",

      ## Composed functions
      "expm1", "fma", "multiply_log", "ldexp", "lmultiply", "log1p",
      "log1m", "log1p_exp", "log1m_exp", "log_diff_exp", "log_mix",
      "log_sum_exp", "log_inv_logit", "log_inv_logit_diff",
      "log1m_inv_logit",

      ## Special functions
      "lambert_w0", "lambert_wm1",

      ## Complex Conversion Functions
      "get_real", "get_imag",

      # Complex-Valued Basic Functions

      ## Complex Construction Functions
      "to_complex",

      # Array Operations

      ## Reductions
      "sum", "prod", "log_sum_exp", "mean", "variance", "sd", "distance",
      "squared_distance", "quantile",

      ## Array size and dimension function
      "dims", "num_elements",

      ## Array broadcasting
      "rep_array",

      ## Array concatenation
      "append_array",

      ## Sorting functions
      "sort_asc", "sort_desc", "sort_indices_asc", "sort_indices_desc",
      "rank",

      ## Reversing functions
      "reverse",

      # Matrix Operations

      ## Integer-valued matrix size functions
      "num_elements", "rows", "cols",

      ## Dot products and specialized products
      "dot_product", "columns_dot_product", "rows_dot_product", "dot_self",
      "columns_dot_self", "rows_dot_self", "tcrossprod", "crossprod",
      "quad_form", "quad_form_diag", "quad_form_sym", "trace_quad_form",
      "trace_gen_quad_form", "multply_lower_tri_self_transpose",
      "diag_pre_multiply", "diag_post_multiply",

      ## Broadcast functions
      "rep_vector", "rep_row_vector", "rep_matrix",
      "symmetrize_from_lower_tri",

      ## Diagonal matrix functions
      "add_diag", "diagonal", "diag_matrix", "identity_matrix",

      ## Container construction functions
      "linspaced_array", "linspaced_int_array", "linspaced_vector",
      "linspaced_row_vector", "one_hot_int_array", "one_hot_array",
      "one_hot_vector", "one_hot_row_vector", "ones_int_array",
      "ones_array", "ones_vector", "ones_row_vector", "zeros_int_array",
      "zeros_array", "zeros_vector", "zeros_row_vector", "uniform_simplex",

      ## Slicing and blocking functions
      "col", "row", "block", "sub_col", "sub_row", "head", "tail",
      "segment",

      ## Matrix concatenation
      "append_col", "append_row",

      ## Special matrix functions
      "softmax", "log_softmax", "cumulative_sum",

      ## Covariance functions
      "cov_exp_quad",

      ## Linear algebra functions and solvers
      "mdivide_left_tri_low", "mdivide_right_tri_low", "mdivide_left_spd",
      "mdivide_right_spd", "matrix_exp", "matrix_exp_multiply",
      "scale_matrix_exp_multiply", "matrix_power", "trace", "determinant",
      "log_determinant", "inverse", "inverse_spd", "chol2inv",
      "generalized_inverse", "eigenvalues_sym", "eigenvectors_sym",
      "qr_thin_Q", "qr_thin_R", "qr_Q", "qr_R", "cholseky_decompose",
      "singular_values", "svd_U", "svd_V",

      # Sparse Matrix Operations

      ## Conversion functions
      "csr_extract_w", "csr_extract_v", "csr_extract_u",
      "csr_to_dense_matrix",

      ## Sparse matrix arithmetic
      "csr_matrix_times_vector",

      # Mixed Operations
      "to_matrix", "to_vector", "to_row_vector", "to_array_2d",
      "to_array_1d",

      # Higher-Order Functions

      ## Algebraic equation solver
      "algebra_solver", "algebra_solver_newton",

      ## Ordinary differential equation
      "ode_rk45", "ode_rk45_tol", "ode_ckrk", "ode_ckrk_tol", "ode_adams",
      "ode_adams_tol", "ode_bdf", "ode_bdf_tol", "ode_adjoint_tol_ctl",

      ## 1D integrator
      "integrate_1d",

      ## Reduce-sum function
      "reduce_sum", "reduce_sum_static",

      ## Map-rect function
      "map_rect",

      # Deprecated Functions
      "integrate_ode_rk45", "integrate_ode", "integrate_ode_adams",
      "integrate_ode_bdf",

      # Hidden Markov Models
      "hmm_marginal", "hmm_latent_rng", "hmm_hidden_state_prob"
    ]

Code snippet

test page

functions {
  array[] real add_array(array[] real x, real y) {
   int n = num_elements(x);
   array[n] z;
   
  for (i in 1:n) {
      z[n] = x[n] + y;
    }

    return z;
    }
}
  data {
    int n;
    array[n] real x;
  }
  parameters {
    real y;
  }
  transformed parameters {
    real z = add(x, y);
  }
  model {
    y ~ std_normal();
  }
}

The BNF grammars page was updated. I see that this is referenced in the prism stan code. The updated file is stan bnf grammars 2.28.

[RunDevelopment]

Ah, that’s what you want to do. That’s pretty easy. I’ll add an alias to those keywords, so you can give them a different color.

[rok-cesnovar]

Thank you so much @RunDevelopment!