Validating financial models


02-Oct-2017 03:02

Then, when the prototype becomes a production system, an attempt is made to perform verification and validation.

It is extremely difficult to perform verification after the fact when normal system development artifacts have not been created and there is no audit trail from concept to product.

The same relations should occur in the model as in the real system.

This technique can be used qualitatively—directions only of outputs—and quantitatively—both directions and (precise) magnitudes of outputs.

Parameters that are sensitive (i.e., cause significant changes in the model's behavior or output) should be made sufficiently accurate prior to using the model.

The model is used to predict (forecast) the system's behavior, and then the system's behavior and the model’s forecast are compared to determine if they are the same.

Typically, government sponsoring organizations that mandate the use of a formal VV&A process do not specify how each phase should be carried out.

Rather, they provide broad guidance that often includes artifacts required from the process.

Verification is an iterative process aimed at determining whether the product of each step in the development of the simulation model fulfills all the requirements levied on it by the previous step and is internally complete, consistent, and correct enough to support the next phase [3].M&S allows decision makers and stakeholders to quantify certain aspects of performance during the system development phase and to provide supplementary data during the testing phase of system acquisition.More important, M&S may play a key role in the qualification ("sell-off") of a system as a means to reduce the cost of a verification test program.Establish quantitative model performance requirements. Often, performance requirements for the domain model are neglected while developing the model.

Complex systems can have interactions that produce unexpected results in seemingly benign situations.Asking individuals knowledgeable about the system whether the model and/or its behavior are reasonable.For example, is the logic in the conceptual model correct and are the model’s input-output relationships reasonable?Here, the development of a simulation model that has undergone a formal verification, validation, and accreditation (VV&A) process is not only desirable, but essential.