SRI FRCEC ( Federal Research Center for Project Evaluation and Consulting Services )

1. SRI FRCEC (Federal Research Center for Project Evaluation and Consulting Services) Mikhail V. Sergeev, Ph.D., Assoc.,+7(495)256-0010 http:\\ www.extech.ru, sergeev@extech.ru TECHNOLOGY OF THE STATE SCIENTIFIC AND TECHNOLOGICAL EXPERT EXAMINATION

2. The scientific and technological expert examination The scientific and technological expert examination is a specific kind of scientific activities, the results of which are subject to special requirements, the main among them being the integrity and objectivity. Сlassification State Expert Examination (SE)or Non-state Expert Examination (NE) VIP –projects, multifaceted projects, Serial (rating)scientific and technological expert examination etc.

3. According to the Federal Law dated 23.08.1996 N 127- FZ (as amended on 07.05.2013) “About Science and State Science and Technology Policy" (v. 14), scientific and technological expert examination has to be organized by the state authorities of the Russian Federation and the bodies of state power of the Subjects of the Russian Federation. - Selection of priority areas of state science and technology policy, and the development of science and technology,- The formation of scientific and technological programs and projects,- Tenders to participate in the scientific and technological programs and projects, monitoring their implementation and the use of scientific and (or) the scientific and technological results in the economy of the state. The expert examination is conducted by organizations performing independent examination, other organizations, as well as individual experts with the participation of representatives of the Authorities of the Russian Federation, the organizations funding the research, and (or) scientific and technological work in:

4. - independence and legal protection of members of the expert process in performing their professional activities; - scientific approach , completeness, comprehensiveness and objectivity of the research objects of expert examination, ensuring the validity of the assessment results; - competence and proficiency of those conducting the review; - systematic organization of expert work and its regulations and methodologies; - aspiration to the global level of science and technology, observation of environmental regulations as well as abidance by safety and security requirements, mandatory compliance with the requirements of national and international standards. The basic principles of scientific and technological expert examination

5. The costs of the state budget of the Russian Federation on research (for reference) According to a database R & D of results (information, software and technical support for the state account of the results of research, development and civil engineering works executed by FRCCE ResearchInstitute), 83 Russian Federal Authority Organizations acted as customers of R & D for civilian use. Public Procurement for the Ministry of Education and Science of Russia amounts to 90% of the funds for R & D and R & D for civilian use.About 70% of the allocations for science in Russia accounts for government funding - (approximately 2 trn.rubleyover the past 5 years).Over the decade, the size of the state budget, earmarked for research purposes, has increased by 10 (at constant prices).

6. Differences of expert examination 6

7. The Principles of the State scientific and technological expert examination (1 of 3) - The principle of assessment status ( "state" ), as reflected in its title and setting her acts;- The principle of compulsory state examination, providing for the need to carry out the preparation of decisions by government and other agencies;- The principle of repeatability of SE in cases of dispute (conflict) situations, violations of regulatory requirements, the emergence of other unforeseen obstacles;- The principle of independence and legal protection of members of the expert process in the discharge of its functions;- The principle of unity of the legal and institutional framework and methodological examination activities as a system factor contributing to comparability and explainable assessments and conclusions derived from different sources;- The principle of objectivity, scientific validity , the findings of examination of evidence, equating it to a variety of scientific research conducted in a particular situation;- The principle of personnel security of expert work that requires careful selection of applicants for professional fulfillment and its interaction with them in the service installed, contractual and other relations; 7

8. The Principles of the State scientific and technological expert examination (2 of 3) - The principle of "transparency" of expert process at all stages under the conditions and limitations imposed by Legislation of the Russian Federation;- The principle of introducing the necessary safety standards in the study, testing, operation of facilities that have passed SE;- The principle of information and other cooperation examinations of different types (economic, environmental , etc.) in the course of carrying out joint tasks or related expert examinations, as well as other substantive and organizational tasks;- The principle of harmonization of the rules , techniques , display and presentation of the results of examination of its customers and other interested parties in accordance with current (approved) regulations ;- The principle of economical effectiveness assessment, which allows to reach an acceptable ratio of the cost of its organization and conduct with the resulting savings (budget) funds with negative findings regarding the project and the rejection of proposals for further work on it;- The principle of the use of modern information technology and automation of processes of interaction between participants and organizational expert works . 8

9. The Principles of the State scientific and technological expert examination (3 of 3) Despite the declarative the nature of these principles, their acceptance of scientific methodology helps build the expert examination, which has always attracted the attention of researchers, even if it was abstracted from any practice. Deviation from it is ALWAYS leads to a decrease in the "quality" of examination, and the rejection of each of these numerous principles (apparently!) is characterized by their system "punctures"!(It is not difficult to construct a table of the "rejection of the principle" and => "negative consequences"!) 9

10. The main technological parameters of the State scientific and technological expert examination - Method of obtaining an expert opinion;- Qualitative and quantitative composition of the experts involved in the event;- the composition of the background information, the initiating peer procedure;- criteria under evaluation of expertized material;- regulations of the expert examination as a whole and the collection of expert opinions in particular;- processing method of expert opinion in order to create expert conclusion;- level of methodological, organizational, informational, and technical support expertise processes. 10

11. The technological cycles of the State scientific and technological expert examination(1 из 5) Reception and registration documents for the facility subject to examination.The resolution guide.Forming the package of documents for examination - replicating materials.Analysis, classification and identification of the object.Formation of the terms of reference for the examination.Information search.Selection of experts.The experts' work – on the subject.Processing and finalization of expert opinions.Preparation and approval of the state examination.Backup.Sending materials of expert examination to the Customer. 11

12. The technological cycles of the State scientific and technological expert examination (2 из 5) Reception and registration documents for the facility subject to examination.Documents on the object to be examined arrive electronically and on paper. The structure of documents includes:- A description of the case- The task (specifications) for examination,- Application (cover letter) for examination.Among the facilities is the passport, which at least contains the name and type of details that provide clerical work at various stages of assessment, in particular, the registration number, the name and code number that uniquely identifies the object as well as the data determining at what stage is the examination of the object and specific data for that stage. The resolution guide.The leadership issues the decision for the examination, or responds to the request with a motivated rejection of examination. In any case, the object is assigned action officer CE, responsible for the organization of the examination. In addition, the solution determines the mode of examination (for large projects, normal, fast, cito!) And as a result, the schedule of its execution. The term is generally from 5 days to a month (for large projects - ½ to 2 months!) . Forming the package of documents for examination - replicating materials.At this stage is adopted the paperless technology associated with copying files to a folder with the materials of the object, and in case of absence of an electronic version, the information on the subject is committed to paper - with the advanced information transfer on storage media. 12

13. The technological cycles of the State scientific and technological expert examination (3 из 5) Analysis, classification and identification of the object.There is a need to classify the object in terms of subject area, its orientation (research & development, industrial application), etc. Identification is to find the prototypes of the object of knowledge, and the like. Formation of the terms of reference for the examination.In general, the task for examination may not contain the specific parameters to be evaluated in the course of expert examination or the Customer may be required to produce a specific set of criteria (novelty, relevance, timing of reality , etc.) of the full range of possible parameters of the object.Expert must have Customer’s Agreement (specified):- Document containing the terms of reference for the examination. Depending on the type of examination the TR may contain a list of questions to be answered, or an expert card of formalized form (template), expert opinion, etc.- Schedule of passing the examination, by the technological process stages or cycles.Information search.Expert administrator is searching for information on the subject of the object in the database sharing, internal database and archive, the Internet, etc. Search results are included in the folder with the materials of the examination object 13

14. The technological cycles of the State scientific and technological expert examination (4 из 5) Selection of experts.Selection of experts involves identifying the range of possible experts on the basis of subject experts of the Federal Roster (performed by the expert administrator) and the appointment of a specific object for examination (performed by the expert and organizer) .This procedure is non-formalized, responsible and is to identify the independent experts in a narrow category of the object of examination.Search of parameters in EDF: the code topics (keywords, specialty codes WAC, GRNTI, OECD, etc.) , region, age , position, experience of the expert (quantity, performance , etc.) , status ( academic degree, academic title, related publications, etc.).The experts are chosen from the list of potential experts on the subject by the group leader (at least 3 - 5), the list composition of subject matter experts is approved by the expert administrator. The experts' work – on the subject.Expert administrator establishes contact (interacts) with the members of the group of subject matter experts, provides (usually remote) reference to each of the groups of subject matter experts for examination, hands over the folder with the materials of the examination object. In addition, it prepares the network part for each group of subject matter experts (subject of the contract, the period of performance, personal data, expert groups, etc.) of the draft contract of civil nature for examination and presents a form of subject matter experts (template) and of expert opinion.Every expert prepares his opinion on the subject on the basis of materials submitted for consideration within the period of time from 2 days to 2 weeks (depending on the type of expert examination) . Processing and finalization of expert opinions .Expert - organizer prepares a consolidated opinion on the subject within the period of time from 3 days to a week. The procedure for each case of expert opinion ( consensus, disagreement, the opposite value) is formalized. Additional subject matter expert examination. 14

15. The technological cycles of the State scientific and technological expert examination (5 из 5) Preparation and approval of the state examination.Expert - The project manager prepares the state examination agreed upon with the participants of the expert process. An additional subject matter expert examination (the procedure needs to be developed).The final version is signed and sent to the Guide by expert-organizer with the covering letter.Backup.A file is created for the object, which includes:- Starting materials- Expert opinions,- The conclusion of the State examination,- A copy of the cover letter to the Customer,- Passport of the object.Folder with the materials of the object is moved to the archive.Sending of expert examination materials to the Customer.The results of the examination (cover letter and the conclusion of the State Examination) are sent to the Customer. 15

16. Experience in organizing and conducting scientific and technological expert examination in SRI FRCCE For the period 1991 - 2013 years the SRI FRCEChas conducted more than 30,000 expert examinations. As shown, the state expert examination in the field of science is an effective tool for improving the efficiency of scientific and technological activities allowing you to select promising ideas, more precisely define the scope of expected financial costs, reduce risks, avoid possible duplication of research.To provide for the organization and holding of SRI FRCCE annually not less than 1000 examinations the standard process of expert examination of programs and projects was established, which was registered in 1992 in ROSSTANDSARD (the State Standard) of Russia as the enterprise standard (STF).An important step in the development of state expert examination in the field of science was the creation by the order of Ministry of Science of Russia ( 1996 ) of the Federal Roster of Experts in scientific and technological sphere containing at present more than 5 thousand scientists and experts from all regions of Russia and covering about 500 research areas.On the basis of modern information technology and network communications in 2010 the SRI FRCCE a system of remote expert examination, which enables us to provide highly qualified expert services in virtually all regions of the Russian Federation, for enterprises and organizations of different sectors of the economy .In connection with the adoption of the Federal Law “On Technical Regulation” (№ 184- FZ of December 27, 2002) a standard process of expert examination was established in SRI FRCCE in the form of organization standards (SRT), which meets all requirements of the organization and conduct of state expert examination in the field of science. 16

17. Some of the results of scientific and technological expert examination of innovative projects eligible for state aid Percentage of good projects 17

18. THANK YOU FOR YOUR ATTENTION! SRI FRCCE R&D Institute (Federal Research Center for Project Evaluation and Consulting Services) Mikhail V. Sergeev, Ph.D., Assoc.,+7(495)256-0010 http:\\ www.extech.ru, sergeev@extech.ru