<<back to publications<< …....... ..                .......>> to Teach0logy.xyz>>

The Traditional Approach Adopted by the NSF

does Not Advance the Science of Education

Dr. Valentin Voroshilov, Boston University Physics Department, Boston University, Boston, USA, valbu@bu.edu

Abstract

The National Science Foundation has been established in 1950 with the mission of: “To promote the progress of science; to advance the national health, prosperity, and welfare; and to secure national defense; and for other purposes”. The NSF’s “Procedures Guide” [1] states that “All proposals submitted to NSF are reviewed according to the two merit review criteria: Intellectual Merit and Broader Impacts.” This requirement, however, does not reflect a social nature of such a human practice like education. This paper is to point at the necessity to differentiate between two different types of human practices, such as a scientific research, and social advancement. This differentiation should be reflected when addressing the funding the NSF provides to the most of the grant applications in the field of education.

Key words

The National Science Foundation, scientific research, social advancement projects, funding, human practice, activity theory.

1. Introduction

1.1 Social Context

America faces big challenges in economic, social, and international areas. The Country will NOT be able to address those challenges without highly qualified workforce. However, in the current state, the U.S. system of education does NOT supply to businesses and institutions sufficient volume of graduates, especially within STEM-related fields. The pool of qualified potential recruits for the U.S. Army, Navy, Ari Forces, and Intelligence Services is shrinking [2, 3].

“According to a 2016 survey of 400 employers from across Massachusetts, 75% said that it was difficult to find people with the right skills to hire in Massachusetts.” “Respondents find deficiencies in the readiness of new hires, not just in “applied skills” like teamwork, critical thinking and communications, but also in simple reading, writing, and math.” [4]

“The number of U.S. citizens and permanent residents earning graduate degrees in science and engineering fell 5 percent from its peak in 2008. At the same time, the number of students on temporary visas earning the same degrees soared by 35 percent.” [5]

“Nearly a half of PhD aerospace engineers, over 65% of PhD computer scientists, and nearly 80% of PhD industrial and manufacturing engineers were born abroad.” [6]

Essentially, the Country has become dependent on foreign intellectuals in the way it used to be dependent on foreign oil. This situation represents a threat to the national security of the Country.

The U.S. Department of Education appropriates about 69 billion dollars per a year. Philanthropic and charitable organizations provide close to 500 million dollars a year to support innovations in education. However, those funds are used to solve structural or social problems in education, such as enhancing material infrastructure, teacher professional development, faculty training.

The task of propelling the advancements in the science of education belongs to the National Science Foundation.

1.2. Role of the NSF

The National Science Foundation is one of the most important arms of the U.S. government which is directly responsible for the implementation policies in the fields of science and technology.

The National Science Foundation has been established in 1950 with the mission of: “To promote the progress of science; to advance the national health, prosperity, and welfare; and to secure national defense; and for other purposes”. NSF’s “Procedures Guide” states that “All proposals submitted to NSF are reviewed according to the two merit review criteria: Intellectual Merit and Broader Impacts.”

This sounds very natural for every scientist.

The problem is that many projects in the field of education are social by their nature and do not intended to produce new scientific knowledge.

A general theory of human practice [7 – 10] recognizes three kinds of broad human practices practices/projects with the goal of advancing human life: (a) scientific research - the goal of a scientific research is discovering new knowledge; (b) engineering and art - the goal of an engineering development is building new devices (and systems of devices), the goal of art is bringing/developing artifacts of art; (c) social advancement - the goal of a social advancement project is developing or adopting new collective practice(s) beneficial for a society (new - for the given social group, but may have been used already by other people).

Since all three practices have different goals, they also should be managed differently, and that includes managing the funding of the projects [11].

In the main part of the paper, a specific example of the NSF funded grant, which is social by its nature, but had to be presented as a scientific research, will be discussed, and the following argument will represent an opinion against this practice, which currently is very common [12, 13].

2. Discussion

2.1. General Methodology

As it will be shown later, many projects in the field of education are social by their nature and do not intended to produce new scientific knowledge. However, in order to get funded by the NSF, they have to fulfill demands imposed by “the Intellectual Merit” – hence, they have to be “dressed” as a scientific research.

Every human practice has some elements of a scientific research: when we start a project, we have some understanding of what we want to achieve, and how we want to achieve that (“a hypothesis”, “methods”), and we view how will we assess how close we are to the goal (“assessments”, “measurements”, “results”, “facts”).

But not every question is “a research questions”, not every statement is “a hypothesis”, and not every search is “a scientific research” [14].

A scientific progress is the result of practices when people do something new for a large part of human culture.

A scientific progress is the result of such practices when people are mainly searching for new knowledge. The result of such practice comes from a comparison of the currently established knowledge and the knowledge obtained during a project. When that new knowledge is described, the goal of a scientific project is achieved. However, when a scientific research presents its results, the social impact of that research may be not seen for years to come, or even never, because that is not expected from a scientific discovery.

But the goal of a social project is to make a specific societal change - here and now.

A social progress is the result of innovative practices of people doing something new - for them – which they did not do in the past, even if a similar practice had been used by different people in a different place at a different time. When we want to induce some societal change, we have to initiate and manage a social project.

There are many things in the world which a similar on the outside but very different on the inside, or by their functions, goals, properties. For example, a space shuttle and a fighter jet look very similar, but only one can fly in the outer world (a space shuttle). The difference between a scientific research and a social project is similar to the difference between an archeological excavation and a dig for a treasure chest: they both use some digging, but the goals and the results are very different.

Currently, the representation of a socially oriented project as a fundamental scientific research is a very common practice; and it is based on a common misconception of what a science is.

There is a wide-spread opinion (also held by many people in the field of education [9]) that:

(a) when a person poses a question, and

(b) then describes some steps which would lead to the answer to this question, and

(c) then describes how he or she would assess if the question was answered correctly

– that person conducts a “scientific research”.

In reality, this procedure is most commonly used for achieving a specific social goal.

This procedure is used when a person feels some disconnection between his or her social position and the position the person desires to have. This procedure has been an object of a study of a General Theory of Human Activity (a.k.a. Activity Theory [7 – 10]), which has several different forms, or academic schools, including the one used in the field of a teacher professional development [15].

The difference between a scientific research and a social project is in “what utilizes what”.

In a scientific research, some social activity is being used as a vehicle to obtain new knowledge. In that case, some advancement in some social practice represents a “collateral” result of the research.

In a social project, some scientific knowledge is being used to achieve positive changes in a certain social situation. In this case, some newly recorded knowledge represents a “collateral” result of the project.

There are many instances when objects similar on the outside are very different on the inside by their properties, or functions, or goals. For example, a space shuttle and a fighter jet look very similar, but only one can fly in the outer world (a space shuttle). To illustrate the difference between a scientific research and a social project one can use an analogy: the difference between a scientific research and a social project is similar to the difference between an archeological excavation and a dig for a treasure chest: they both use some digging, but the goals and the results are very much different.

Many of the projects “imposed” on teachers are social projects by their nature, and should be treated and managed as such.

2.2. Specific Analysis

Below is an excerpt from an abstract of a grant proposal of a certain university (the grant received an award from the NSF): “University are conducting research on the relationship between mathematical knowledge for teaching (MKT), teaching practice, and student outcomes. … The research questions are as follows. How effective is Math Solutions as compared to a typical ad-hoc mathematics professional development? Does Math Solutions improve teachers’ MKT, the quality of their instruction, and/or their students’ outcomes? How are different aspects of teachers’ mathematical knowledge and instructions related to student achievement?”

Anyone who is reading the abstract will immediately understand that the project is about professional development of math teachers. The ultimate goal of the project is to train 80 fourth and fifth grade teachers.

There is no doubt that the trainers need to know if the training process they plan to use will positively affect the math skills and math knowledge of the trainees. However, the main goal of the project, which is improving content knowledge of math teachers, does not represent a scientific problem which would require a scientific research.

This grant proposal represents a clear example of a social project with the goal of advancing math preparation of school teachers. But in order to get the funding from the NSF, grantees had to make it seen as a scientific project.

The research question (“How effective is Math Solutions as compared to a typical ad-hoc mathematics professional development?”) is irrelevant. All possible teacher professional development programs should be available to teachers.

As long as teachers will be able to “vote with feet” there is no need to “scientifically” research which program is better (assuming that the NSF will be keeping track of different programs and helping teachers to find the information on the best programs).

Instead of a “research question” applicants should have formulated “a social goal”.

This is what the grantees should had written: “We want to teach mathematics to 80 teachers; this is a description of what the teachers will learn, and this is a description of how we will assess the results; and for that we need 4.7 million dollars for five years (which is close to sixty thousand dollars per a preparation of a single teacher)”.

In 2016, the NSF awarded close to $61 million in new projects “to enhance understanding of STEM education and workforce development.

For example, the NSF’s website states the following (in part) [16]:

“The new awards fund projects aimed at generating foundational knowledge in:

_ Improving and advancing STEM learning and learning environments for students, parents, teachers and the general population in all settings, from formal and informal education to technological learning environments.

_ Supporting and preparing a STEM professional workforce that is ready to capitalize on unprecedented advances in technology and science and address current and future global, social and economic challenges.

_ Diversifying and increasing participation in STEM, effectively building institutional capacity and informal learning environments that foster the untapped potential of underrepresented groups in STEM fields.”

The brief reading of the bullets already raises a question – do the goals really represent the search for a fundamental scientific knowledge, or they rather aim at improving immediate social issues education system currently deals with?

The next web page leads to “the complete list of ECR projects and their abstracts” [17].

The total number of projects funded within $61 million is 114.

Below are the titles of the awards listed on the first page (out of four).

(1). “Transitioning Learners to Calculus In Community Colleges (TLC3): Advancing Strategies for Success In STEM”.

(2). “ArguLex - Applying Automated Analysis to a Learning Progression for Argumentation”.

(3). “Partnership for Building Capacity for Improvement In State Science Education”

(4). “EAGER: Early Stage Research on Automatically Identifying Instructional Moves in Mathematics”

(5). “Proportions Playground: A Dynamic World to Support Teachers' Proportional Reasoning”

(6). “Domain-General and Domain-Specific Training to Improve Children's Mathematics”

(7). “Transitioning Learners to Calculus in Community Colleges (TLC3): Advancing Strategies for Success in STEM”

(8). “Transitioning learners to Calculus in Community Colleges (TLC3): Advancing Strategies for Success in STEM”

(9). “Transitioning Learners to Calculus in Community Colleges (TLC3): Advancing Strategies for Success in STEM”

(10). “NCS-FO: Integrative Knowledge Modeling in Cognitive Neuroimaging”

(11). “NCS-FO: Learning Efficient Visual Representations From Realistic Environments Across Time Scales”

One can see, that the majority of the 114 projects funded by the NSF aims at the achievement of some positive social changes in a certain educational environment.

For example, the very first project at the top of the first page “Transitioning Learners to Calculus in Community Colleges” aims at “Improving student outcomes in mathematics courses in community colleges”.

The main vehicle of the project is improving instructions by utilizing various instruments (mostly surveys, and self-assessments). There is no doubt that this is an important social project. However, tis does not represent a fundamental scientific research.

And many more projects sound like this one.

If we strip off all the scientific language, we will read – paraphrasing –

1) “We want our students to do better. For that we plan on trying this.” – if the project mostly involves faculty or teachers who directly teach students.

or

2) “We want our school teachers to teach better. For that we plan on trying this.” – if the project mostly involves faculty from a school of education.

Out of the eleven grants from page one, only the last two can be seen as a scientific research proposals. Totally, only 3 projects from 114 really fall in a category “scientific research”. Those three scientific projects are related to a neurology of thinking; they study various connections between process of thinking and processes happening in a brain while thinking. The total amount of funding set aside for those projects is #2,242,982, which is equal to 3.7 % of the total funds.

It means that 96.3 % of the funds are being used for projects of another kind (do not belong to a fundamental scientific research).

If one just reads the titles of the projects, one can find several more projects which also may be sought as a part of a fundamental scientific research, but that would require the detailed analysis of the projects.

A brief reading of the project titles and of some of the abstracts shows that the majority of the projects are of a social nature; they aim at improving a current social situation by solving a specific immediate social problem within the field of education.

No doubt, some of those socially oriented projects are fundamentally important for making education better, more successful, more student oriented, more diverse.

But they would not help much to advance a science of education.

2.3. Reflection

One might ask, what harm is in calling a social project as a scientific one? Both types are important, and do good for education. Down the road, universities get grants, supposedly teachers get some help with their professional development, and hopefully students become better educated.

A short answer is: it is bad because it makes an impression of a huge amount of a scientific research happening in the field of education; when in fact a true scientific research in the field of education does not exceed 3 – 5 % of the total funding.

If we want to promote a science of education to a true science we need to change that.

The NSF should not expect any big scientific outcomes from any social project, but should be very demanding regarding the methods for evaluating the success of the project. In the case described above, the NSF should had demanded that applicants would guaranty that their professional development approach would definitely and visibly (a.k.a. measurably) “improve teachers’ MKT, the quality of their instruction, and (!) their students’ outcomes” – or money back.

The bigger problem is that currently and unwillingly, the NSF forces innovators (a.k.a. people producing a new social outcome, like math teachers who know math) to make them to look as scientists (a.k.a. people producing a new knowledge, like mathematicians).

The example of a grant proposal used above to demonstrate the focus of this paper is one of many grants which are social by the nature of the goals, but “dressed up” as scientific projects.

That happens because the NSF essentially forces people into faking doing science.

The core of any science is being truthful about everything; including goals, methods, types of actions being used to achieve the goals. If people assume that faking science is fine – even for the sake of achieving positive social changes – that will water down the essence of science. Such an attitude as “I do something good, so what if I pretend that I do science” – if not confined – might spread out into other practices.

K-12 education is not the only one type of education distorted by the demands imposed by the NSF. There are many projects on a college and university levels, which are also social by their nature (i.e. with the main goal of improving some specific features of educational reality), but “dressed” as scientific ones. One of the widely-spread examples is the well-known Learning Assistants program. This program has originated as a means for fixing the shortage of STEM middle and high school teachers. One of the main premises (a.k.a. “research questions”, a.k.a. “hypotheses”) is that when undergraduate students get an opportunity to be immersed into a university teaching process, they will eventually end up teaching middle and high school students. In reality, no more than two percent of Learning Assistants choose to become teachers. The program might eventually have a strong social impact, because what the program really does is generating a growing number of educated and socially active people, who also have elevated awareness of what it means and how it feels to be a teacher. However, this is an example of a social action which does not require any scientific research; because there is no doubt that adding to an undergraduate class of students more people who (in addition to an instructor and graduate teaching assistants) can provide an extra help to students, will result in better learning outcomes (on average).

In fact, almost every educational “research” project at a college or a university level is not a science project, but the one designed to keep faculty trying new things in the way faculty teaches. I say, ninety percent of issues with education college students has its roots in a high and middles school – this is where ninety percent of funding should go; when ninety percent of high school graduates will have the background sufficient to study at a college level without using any remedial courses, the most of the issues related to college education will just disappear. What really needs funding is developing standard procedures for measuring learning outcomes of college students, which (procedures) would be used by ALL colleges across the country. Until this task is finished, ANY educational project at a college or university level will not be scientific. Until standard procedures for measuring learning outcomes of college students will be developed and adopted, all college level projects will be just dressed as a “research”.

3. Conclusion:

By making social projects to look like scientific ones, the NSF is working against its own mission by helping developing wrong attitude about science – at least within a large number of projects in the field of education.

It is a well-established fact that both, the Religion and the Government, have benefited from the separation of Church and State.

Similarly, the separation of programs for social advancement from programs for scientific advancement will be beneficial for both, social and scientific advancement.

As any well-established science, the science of education needs development of research facilities specifically designed and designated to study phenomena within learning and teaching processes [18]. The NSF should adopt to the development of the science of education the approach which proved to be effective for such large scientific programs as the “Manhattan Project’, or the “Apollo Program”.

References:

[1]. The NSF’s “Procedures Guide”, https://www.nsf.gov/publications/pub_summ.jsp?ods_key=gpg

[2]. “Military turns down 80 percent of applicants”, Reuters, https://www.rt.com/usa/158992-military-80-percent-rejection-rate/

[3]. “A few good men, women … anybody?”, ForeignPolicy.com, http://foreignpolicy.com/2015/04/09/a-few-good-men-women-anybody-finding-qualified-recruits-becoming-more-difficult-for-the-militarys-combat-forces/

[4]. 2016 MassINC Polling Group report, http://www.mbae.org/wp-content/uploads/2016/11/FINAL-Report-2016-MBAE-Employer-Poll-for-web.pdf

[5]. US News, http://www.usnews.com/news/articles/2016-05-17/more-stem-degrees-going-to-foreign-students

[6]. American Physical Society, http://www.aps.org/policy/reports/popa-reports/upload/POPASTEMReport.pdf

[7]. Shchedrovitsky, G.P., Systems Research, II. Methodological Problems. Edited by J.M. Gvishiani. Pergamon Press, 1985.

[8]. Engeström, Y., Miettinen, R. & Punamäki, R-L. (Eds.) (1999). Perspectives on activity theory. Cambridge: Cambridge University Press.

[9]. Yrjö Engeström, “From Teams to Knots: Activity-Theoretical Studies of Collaboration and Learning at Work (Learning in Doing: Social, Cognitive and Computational Perspectives)”, Cambridge University Press, 2008.

[10]. Hasan, H. & Kazlauskas, A., “Activity Theory: who is doing what, why and how.” In H. Hasan (Eds.), Being Practical with Theory: A Window into Business Research (pp. 9-14). Wollongong, Australia, 2014.

[11]. “Research funding: the problems with priorities”, Editorial. Nature Materials, 2, 639, 2003.

[12]. Fang, H., "Peer review and over-competitive research funding fostering mainstream opinion to monopoly". Scientometrics. 87 (2): 293–301, 2011.

[13]. Julia Belluz, Brad Plumer, Brain Resnick, “The 7 biggest problems facing science”, VOX, 2016, http://www.vox.com/2016/7/14/12016710/science-challeges-research-funding-peer-review-process.

[14]. Valentin Voroshilov, “Critical Reading of “Making Sense of Confusion” by Jason E. Dowd, Ives Araujo, and Eric Mazur”.

[15]. Valentin Voroshilov, “Professional Designing as One of Key Competencies of Modern Teacher”, In book “Facilitating In-Service Teacher Training for Professional Development”, IGI Clobal, 2017.

[16]. https://www.nsf.gov/news/news_summ.jsp?cntn_id=190509&WT.mc_id=USNSF_51&WT.mc_ev=click

[17]. https://www.nsf.gov/awardsearch/advancedSearchResult?PIId=&PIFirstName=&PILastName=&PIOrganization=&PIState=&PIZip=&PICountry=&ProgOrganization=&ProgEleCode=7980&BooleanElement=All&ProgRefCode=&BooleanRef=All&Program=&ProgOfficer=&Keyword=&AwardNumberOperator=&AwardAmount=&AwardInstrument=&ActiveAwards=true&OriginalAwardDateOperator=Range&OriginalAwardDateFrom=10%2F01%2F2015&OriginalAwardDateTo=09%2F30%2F2016&StartDateOperator=&ExpDateOperator

[18]. Valentin Voroshilov, “What Infrastructure Do We Need to Build to Promote Education Research to a True Science?”, http://www.teachology.xyz/30uS.html.

********************

These pictures, naturally, are not a part of the paper; they have been borrowed from older web-posts as an illustration to this paper.

<<back to publications<< …....... ..                .......>> to Teach0logy.xyz>>