Evaluation – tests, assignments + user testing

Hello 🙂

I tried to rewrite my module about mind maps to Umbraco, so here it is.

I created a test in IS (it´s also in my module), I think now it´s very easy for me because I create tests in IS for KPI22. But it the beginning it was quite hard. I use text editor for those questions and it´s quite quick when you know what type of question to use. 🙂

I hope my assignment is OK, but I am not sure about those instructions for the tutor. 😀 It´s quite difficult to do it for mind maps because every mind map is special and every technique for creating mind maps is individual and unique. I hope you don´t mind it´s in Czech, because originally I put it in Umbraco module which is in Czech, but then Ondřej suggested it would be better to put it here on the web. 🙂 So here are my instructions.


Instrukce pro tutora: 

  • Motivovat studenty, aby se toho nebáli a ukázat jim, jak se mapa tvoří na nějakém základním příkladě. (Ten asi ještě doplním do úkolu na web.)
  • Je třeba si uvědomit, že každý student úkol může pojmout jinak, každý je jinak kreativní a jinak myslí.
  • Jádro mapy by mělo být uprostřed.
  • Student by měl postupovat od obecného ke konkrétnímu.
  • Student by neměl používat věty nebo příliš dlouhá hesla.
  • Mapa by měla dávat smysl, hesla v rámci větví by na sebe měla navazovat.
  • (Mapa by měla obsahovat alespoň 3 hlavní větve.)
  • (Hloubka větví by měla být alespoň 3.)

I´m sorry I didn´t create user testing scenario yet, but I´m not quite sure what to do and what exactly does it mean for us. How will it look like etc. 😀 I wasn´t there on the 24th… 😀


So I did a few changes, actually I deleted a few thoughts after Joseph gave me his advice (thank you 🙂 ). Below you can see my previous concerns, cause I couldn´t decide what to do. 😀

Pre-scenario

This is my pre-scenario and I am not quite sure about it. Feel free to give me any advice and comments. 🙂

The main obstacle for me is was to decide these ideas:

  • Should I give students feedback and they will be giving me feedback continuously (after they finish reading module, after test and after assignment)? Or after all the testing?
  •  Should I do it all online (Skype) or „offline“ (café, library). I think it will be better „offline“. They can show me exactly what was wrong, what did they like or not etc.
  • Should I be there? I mean, should I sit next to the student when he/she´s reading it, doing test and assignment? What if they will need more time for it then just one or two hours in a coffee house? They may be nervous when having me next to them… I think it would be better to send them the module and let them do the test at home and then maybe do the assignment somehow together?
  • Should I ask them what do they remember? I think the test does it already.

Here is my final pre-scenario:

  1. Ask at least two students to co-operate. I should find someone who doesn´t know anything (or little) about mind maps and who doesn´t use it. –> Pre-questionsHave they ever heard about mind maps? If yes, what. If not, what do they think it is. 
  2. Send them the link to my module which I hope they´ll read. 😀
  3. Ask them now what is their opinion about the module? Or at the end? –> What would they change, what did they like or not, what did they have a problem with, where were they stuck, did they understand all of it, did they click on those links inside, how did they feel, did they really read everything or did they skip something?
  4. Students will try the test.
  5. Ask them about it? Or at the end? –> Did they understand the questions? Was it hard or easy? What would they change? What types of questions would they prefer? Is the test even useful for them? Do they find it useful? 
  6. Students will do the assignment. –> Alone at home? Or with me next to them in a coffee house or somewhere? Should I help them with that? Or let them do it alone and then give them feedback?
  7. Give feedback. –> Online or offline? I think offline will be better and maybe even better for me and more useful for the student.
  8. Ask them about the assignment. (Or the whole user testing) –> Did they understand it? Did the get from the text what should they do? Was it hard or easy? What would they change? Did they like it or not? Why? Would they like to try something different (suggestion to another assignment)? What types of assignments would they prefer? Was the assignment useful for them? Do they find it useful? 
  9. Post-questions: What do they think about mind maps now after the user testing? Is it useful? Is it efficient? Is it fun? What adjectives would they connect to mind mapping? Are they willing to use it? If yes, why? If not, why? What would they prefer – paper or tool? 

I put those questions together in Czech here. 🙂

 

8 thoughts on “Evaluation – tests, assignments + user testing

  1. Hi Erika!

    good job with the updated module in Umbraco. 🙂 It’s very clear and it says the important things.
    I think you understood well the instructions for the tutor, maybe I wouldn’t include it in the module (as the module is supposed to be for the student) and put it just here in this blog post.

    About the assignment – I think your assignment is good, I would maybe add the steps about how to do it, but in the other hand, the steps are almost written in your module. So maybe I would just put there text like: “Follow the steps from the module in part called Jaké jsou zásady a tipy pro tvorbu myšlenkových map?”… or something like that.
    Or maybe create new steps (just the most important steps which the student should follow). It’s the thing which we were talking about in the last lesson so I understand you didn’t know as you haven’t been there. 🙂

    About the test – I like it! It’s not too easy and also not too difficult if the student has read your module. I have made 1 mistake in the last question… 😀

    How long time it took you to think up the questions? I’m just wondering because I had a bit problem with that because every time I created something too easy or something which haven’t been necessary for my objectives…so it took me a lot of rewriting and deleting.

    Liked by 2 people

    1. Hello Ondřej,

      thank you for your feedback, I´ll try to rewrite the assignment a little bit. 🙂 And you´re right, I should put it hose instructions here on the web, not in Umbraco.

      And about the questions, well actually me and my friend we create questions for KPI22, so I used three questions from that and I rewrote them a little bit. Then I created two other questions. I think it took me about 30 minutes and putting it into IS was quick, because I already knew how to do it. But actually, it was quite hard for me, too, because the module is not so long and mind mapping is quite individual.

      Liked by 1 person

  2. Hi! I like the module as well 🙂 it is clear and all the important is mentioned. Also, test questions are totally ok, you used different types of Qs and I think they are all relative. The assignment is cool, but I think that some of the tutor´s advice should be in the instructions for a student (like 3 branches etc.), so he/she would know how precise the task should be. 🙂

    Liked by 2 people

    1. Hello Hanka,

      thank you for your feedback. Actually, I was thinking about putting a few instructions into the assignment for students so I will rewrite it. 🙂

      Liked by 1 person

  3. Hi Erika,

    thank you for nice job 🙂 Just a few recommendations or comments…

    Last time I sad for me creating test questions is one of the hardest things – you have to ask yourself do they really need to remember this information? Do they need to remember Buzan’s name or just know how to use mind map? How to make a question clear, challenging, but not too hard at the same time?
    Eg. 3. Co nepatří mezí výhody myšlenkových map?– Jsou přehledné. – Pomáhají nám si věci lépe zapamatovat. – Mohou nám pomoci se lépe učit. – Hodí se spíše pro lidi, kteří se učí vizuálně.
    I know what you mean, but actually, every option is questionable, what do you think? 🙂 Try to avoid NEpatří negative formulation.

    So, me as your tutor, how should I know what is the bad map? When I should let the student make a correction? Should I count branches? How should I know he really started in the centre? 🙂

    Liked by 1 person

    1. Hello Hanka,

      thank you very much. I´ll try to recreate those questions a little bit. 🙂

      And you´re right about those instructions for the tutor… But still, I am not sure how to evaluate mind maps. When you don´t understand student´s map, you can´t say it´s wrong, because every map is special. I think you should correct student when you clearly see he didn’t catch the point, he uses long sentences and the words are not connected somehow. So I guess you don´t have to count the branches… yeah… 😀

      And about the center of the map – I thought that the center of the map should be in the center, I mean, not in the top right corner 😀

      Like

  4. Hi Erika,

    I missed last hour, so I look at your job know and it is pretty cool.

    I have for you few recommendation and tips for your user testing.
    First of all, you should define, what you need to figure out – Is my module intelligible? Do understand students the task? Etc.

    Online vs. offline – It does not matter. If you are going to testing online, you should share the screen (for example the tool appear.in is good for it). If you are going to testing offline, you will look at the screen.

    Point 3.
    „What would they change“ – it is not a good question – They will invent a solution – that’s your job. You know from other questions – what did they have a problem with? what did they like? where were they stuck? what’s the fun? etc.

    Ask them about that after reading.
    They can study your module 2 hours – if they will be under time pressure, you can reveal more things.

    Point 6 – it is better if you can look at their work – and they will comment, what they actually do. Tip method: Myšlení nahlas.

    Should I help them with that? – nope, you can see if is there some problem with your assignment.

    I think you’re thinking about testing well enough! All you have to do is sort a few questions and just choose one of the variants.

    If you have any questions, you can write me down – I’m misinterpreting in English. Perhaps I helped a bit 🙂 Good luck!

    Liked by 2 people

    1. Hello Joseph, thank you very much for your advice. 🙂 I´ll do is as you said. You were totally right about that question what would they change. That was stupid! 😀

      Like

Comments are closed.