Is this normal??

My bf and I have been dating 10 mos. He is divorced with 2 young children. He and his ex are waiting for the right time to integrate them into our relationship--which I totally respect. He spends every day with them from after work (5pm) until 8pm and then comes to my house (he basically lives with me). He also has them every weekend, sometimes over night on the weekend. I am very understanding that he loves his kids so much and wants to see them but the only time we have a date (unless its spending the night together) is if the ex wants the kids which is rarely because they sleep at her place so her free time is every day when my bf gets them. He says he picks them up from her place every day and takes them to dinner and to play but (admittedly I feel selfish about this) I am starting to question this behavior because it has been 10 mos every single day so we can never plan and I am not included w/ them. Should I be concerned?