teaching - In case of in-lecture quizzes, is it unreasonable to fail students who are late or absent?

I am teaching a large undergraduate class this semester. One day a week (same day every week) we have a quiz. The syllabus doesn't specify at what moment of the class period the quiz will be given. It does say though that students who missed the quiz aren't allowed to make it up. So, I've had the following situation since the semester started. First and second week I gave the quiz at the end of the class. During week 3 I already had a few students coming more than 20 minutes late, but early enough not to miss the 15 min quiz, which I gave again at the end. So, this week (week 4) I gave the quiz 10 minutes after the start of the class. The actual time of the quiz wasn't announced in advance. And yes, I've had many students coming late and some came when the quiz was already over. These students couldn't have 15 minutes like others if any time at all. Now these students are claiming that I needed to tell them in advance at what time the quiz was supposed to be or emphasize that it can be at any time, or otherwise they follow the pattern.

All students who came in late were the ones who were late the previous week except one student.

I repeat that it's a big lecture hall, and I don't want to take notes when a specific student actually started his or her quiz and add 15 minutes to that time. Also, after quiz I want all of them to concentrate on the class material. And I don't want to give it at the end all the time as some students will come in late, and that disrupts the flow of a lecture, distracts other students and generally erodes class morale.

I drop quite a lot of lowest quiz grades at the end of the semester to meet those cases when students do have to be absent. So, a good student's grade won't be affected if he or she will miss a couple of quizzes.

I am currently facing lots of negative emails from students who were late and didn't get the grade they would get if they had the same time as everybody else. Do you think it's me being unreasonable? Their main argument is that it should be clearly stated when a quiz is given or emphasized that it can be given at any time, which I find unnecessary.