Tuesday, December 4, 2007

Session 4: Reflections on Assessment Tools

The use of WIHIC provides teachers with a way of looking at different aspects of their classroom environment which they intended to create. By getting the responses of the students, one can first and foremost see what the students see as the most ideal environment for them to thrive in. After administering the actual form, one can compare the difference between the actual and ideal. This use of comparing preferred and actual items is an excellent idea as teachers can really see how far the students are from what they perceive they would like to be at. As such, teachers need not be always wondering how much more they more to improve to meet the students' expectations or that they are contented with so-called positive results although these results may still be less than what they deemed as ideal.

Take for example if the class actual ideal has a mean of 3 (1 being least ideal and 4 being most ideal), one more conclude that the class is doing well and not much improvement needs to be put in. However if the class preferred happen to be 3.75, then although the actual is positive, it still shows that there is some way to go. A so-called negative result may also not be too bad. If the actual mean is 2.5, one may think it's bad. But if the preferred is 2.75, then the disparity is not that far. So the use of the actual and preferred questionnaire is helpful in data analysis.

As for the creation of the items within a questionnaire, much thought is needed to create a form that is useful for analysis. This has to be done firstly to ensure that what needs to be measured is being measured. Dimensions to include may be based on previous research papers that have been published to form the basis for adaptation. The other aspect would be to make sure that the questionnaire is in itself a reliable and valid instrument so that results and recommendations made after analysis are actually what can improve the current situation rather than wrongly interpreted results and recommendations that will later on be carried out but have no significant impact whatsoever.

This is absolutely necessary for viable Action Research topics to enable a teacher to go through at least two cycles of the same topic. This leads to the question of what type of Action research are we conducting in schools. Much of the time, these are done haphazardly due to the lack of time even though the finished article may look quite polished. Topics are chosen not for the purpose of doing more than one cycle but rather something that seems interesting at the moment. With real follow-up missing, the true essence of AR is slightly lost. For AR to thrive in Singapore, it's not just about having school leaders push for it. It must be marketed as being helpful to one's teaching practice and on a longer term basis (at least 2 cycles) and more help in terms of actual design can be provided. Of course, schools already engage consultants to facilitate the beginning process, which may include the basic research design and methodology, and also occasionally in the interpretation of results. However the "problem" begins much earlier in that these researchers are often not prepared to perform it.

So in conducting quantitative research especially for AR, more thought can be put in to include instruments such as WIHIC for the purpose of improving classroom processes and not just to get a snapshot of what is happening.

Session 5: Analysis of results (with SPSS help)

Things to look out for:

Reliability Cronbach alpha: >0.5

Cronbach alpha reliability (With example) should be above 0.5 to about 0.7 for an item to be deemed as reliability. One should note that too high an output (say 0.95 - 1.0), this may mean that this item is most likely redundant.

Inter-item Correlation Matrix
1.0 for same item; the higher the value for an item, the more correlated the two items are

Paired Sample T-test
When choosing 95% Confidence Interval, grouped-means items which are seen to be less than 0.05 are deemed to be significant and have to be addressed.

To calculate basic information in SPSS

Analyse -> Scale -> Reliability Analysis
Select the variables needed
Click Statistics and select
  • Descriptives (Item, Scale, Scale if item deleted),
  • Inter-Items (Correlation, Covariance),
  • Summaries (Mean, Variances, Covariance, Correlation),
  • Hotelling's T-Squared Test

To calculate T-test

Analyse->Compare Means->Paired Sample t-test

Session 4: Case Study Slides

Session 4: Instrumentation Issues

  • Type of questions asked (dimensions)
  • Number of questions
  • Readability
  • How the choices are placed (putting SA as the 1st choice helps people be more positive towards giving better responses)
  • How respondents response (shade, tick...)

In the area of dimensions we can consider the School Level Environment Questionnaire(SLEQ) .

According to Moo's Scheme:

Relationship

  • Student support
  • Affiliation

Personal development

  • Professional interest
  • Achievement orientation

System change/Maintenance

  • Staff freedom
  • Participatory decision making
  • Innovation
  • Resource adequacy
  • Work pressure

Monday, December 3, 2007

Session 3: Takeaways

It's interesting to note that although there are lost of tools available for teachers to assess their own classroom environment, there is a lack of it's actual usage amongst practitioners. Some reasons for not doing so include:
  • Fear of knowing what is really happening
  • Would like to know but have not been furnished with relevant information
  • Have knowledge of known tools but have inadequate know-how of interpreting results

These can be slowly overcome by providing teachers with how these can be used for their own practice and with the background of noting the linkages between a positive classroom climate with positive academic achievement, it may help others to jump onto the bandwagon.

Of course an often (mis)quoted factor of not doing such studies is the lack of time. Teachers have to come to a point where they view the importance of their contribution within the class and by finding out their strengths and weaknesses, evaluate further and deeper so that there can be become even better practitioners.

Would I use any of the tools mentioned? Certainly worth a try for such knowledge will help to bridge the gap between the teacher's perception and the student's perception. It is in being open-minded that students will unwittingly become closer to the teacher that they evaluate if they can see that the teacher has been trying to change to fit their style of learning. For teachers must model a real desire to learn in order that students themselves have an example to follow. And what better way then to engage in a dialogue with them.

Take time to know them and also in the process to know oneself. For learning is not just about understanding a discipline or the subject matter; it is also about the discovery of our talents and our personality. It's about how we react in different ways to different situations. For assessment tools can provide statistically results that may make meaning for a dissertation but have no real impact on improvements to better oneself. Didactic teaching, student-centred learning, problem-based learning, case study learning... all are buzzwords that are created but no one pedagogy is able to provide everything that a child needs. A range of pedagogy must be adopted for students to learn as well as experience a part of the real world environment in a safe setting.

Know thyself and be ready to embrace change!

-- Dr Quek's Online Notes

Session 3: A Short History of Learning Environment Research - Murray

1930s - Murray's need-press model and Lewin's formula

Henry A. Murray (1893 - 1988) was an American psychologist who majored in history at Harvard University and earned an M.D and an M.A in biology in 1919 from Columbia University.

In 1927, he developed the concept of latent (inner) needs , manifest (outward) needs, "press" (external influences on motivation) and "thema" - "a pattern of press and need that coalesces around particular interactions".

In 1938 he published Explorations in Personality, now a classic in psychology, which includes a description of the Thematic Apperception Test (TAT) , manual. "Apperception" refers to projecting fantasy imagery onto an objective stimulus. This allowed trained interpreters to reveal a person's dominant drives, emotions, sentiments, complexes and conflicts of a personality. This was part of Murray's Psychogenic Needs which included the Need for Power (nPow), Affliction (nAff) and Achievement (nAch). This formed the basis of models of management effectiveness (Richard Boyatzis), Maslow's hierarchy of needs, and ideas relating to Positive psychology.

Some Sample TAT Cards

Slides on his life can be found here.

A major assumption of Murray’s theory was that behaviour is driven by an internal state of disequilibrium. We will behave in a certain fashion till we reach equilibrium.

Some reknown works include:
(1938). Explorations in Personality. New York: Oxford University Press
(1940). What should psychologists do about psychoanalysis? Journal of Abnormal and Social Psychology, 35, 150–175.
(1953) Personality in nature, society, and culture
(1962) Explorations in Personality: A Clinical and Experimental Study of Fifty Men of College Age


Other famous researchers...


1960s - Walberg's and Moo's developed LEI and CES

Moo's schema:
Personal Development: (Personal Growth & Self-enhancement)
Stimulation - Pleasure & Satisfaction,
Achievement - Improvement of performance, progression & broadening of interests
Efficacy - Feeling of competence, ability and success

Relationships: (Extent of people involvement & support for each other)
Helpfulness - Cooperation and support among students,
Participation - Participation and valuation of each individual in the group,
Responsibility - Trust and responsibility of activities and learning

System change/maintenance: (orderliness of environment)
Creativity - Personal expression with creative and artistic activities,
Influence - Expression of views and ideas on school matters and opportunity to be heard,
Safety - Absence of danger, violence and harassment,
Control - Presence of rules, organisation, self-discipline

1970s - Fraser and Fisher

Session 3: What to Assess?

Physical - seating arrangements, noise level, space, ventilation, resources

Students - group work, co-operative behaviour, group sizes, concentration level

Social - Teacher's modes of instruction, acceptance/rejection, educational tasks required, activities