Highlights: Twitter Spaces 8/17
Pro bettors Michael Craig, Eddie Walls discuss modeling, among other topics
Professional bettors Michael Craig and Eddie Walls hosted a roughly 45-minute Twitter Spaces Wednesday evening. They discussed a variety of topics, then took some questions, but opened the show with a discussion about modeling and power ratings. What does a model hope to accomplish? What are some ways a model can be utilized? How do you know if a model is any good? How do you make adjustments to certain teams the model can’t seem to get right? How do you make subjective adjustments to a team’s power rating as the season progresses? In college football specifically, how do you account for the transfer portal? About midway through the show, Eddie also gave out a few college football season win totals plays.
We think you’ll enjoy the full listen. Click here for a show replay.
A partial transcript on modeling and power ratings is below.
Eddie Walls: I’m not a modeler. People who know me know that. So why don’t we start with you telling us what a model does, what its function is, and how you use it.
Michael Craig: Well, the simplest way to say it is that a model creates a score projection for games. It takes in a lot of input variables—it can take in as many as you want—and basically using whatever algorithm or math you have, you use those numbers and stats to make a score projection for games. That’s what I do. I’m talking a ‘bottom-up approach’ when I say that. I guess ‘top down’ stuff I’m kind of making lines on derivative markets, from the closing line for the game or what I think will be the closing line for the game. Those are the two ways that I would model. If you don’t model though, you’re doing some sort of numbers, some sort of power ratings, right?
Eddie: Correct, I make a power rating for every single team. My process is obviously through pen and paper. The more I learn about a team, the more I can find a number I feel good about. I try to make a number very, very early in the offseason just based off last year’s priors, and then it gets better throughout the summer. Then by July, I start to feel pretty comfortable. But, can you tell me, and this is the biggest thing I always worry about: What do you do if a model is wrong? Like, I hear these stories, someone says their model failed, they had way too many plays that didn’t make sense. What are some of the pros and cons of using a model in your view?
Mike: Well, I think when you start a model, you’re looking for ways to convert whatever raw stats or adjusted stats you have into a score projection or whatever you’re going to use them for. One important thing is you don’t want to use the same data to create the model. You use certain data to create the model, and then other data to back test the model. So, I think that’s one common mistake people make. If you use data to create a model, and then you back test using that same data, it’s going to show it’s a good model. So that’s one potential mistake a lot of people can make is using the same data and back testing the same data because it’s going to come out better than it actually is. Now, in terms of good model or bad model, I’ve created plenty of bad models. Hopefully I’ve backtested them enough against other data that shows me that I don’t have anything that’s good. I’ve always wanted to model hockey, but I’ve never gotten anything that would work. Just an example, back in the day I created something and the first year I backtested it, it was plus 40-50 units for a season. I was really excited. But then I backtested the two seasons before that and it was like minus-30 units and minus-20 units. So, net across the three years, it was nothing. So I think it’s important that before you put a model into play and risk real money on it, that you do the due diligence to make sure you have something that’s worthwhile to put out there. I did want to shout out Waz, from BettorIQ, who had a good tweet on this. He wrote:
The sports betting landscape has started to shift. The edge for modelers as a whole has been slowly diminishing over the last 5+ years. Pure handicappers who incorporate more qualitative elements have seen increases in their edge as the quality of information improves. The main reason: all "winning" models eventually converge to the same point (using similar metrics, statistical tools, techniques, etc) and the market becomes much more efficient as a result.
I do think there’s some truth to that. I know through the years, my process has changed. In the past I was solely model based on certain things, and I’m much more subjective now than I’ve been in the past. So I do agree with that. What do you think of that?
Eddie: I don’t know enough about models to act like I have a really good opinion, to be honest. But here’s a question for you, and your model. For college football, how does the transfer portal shift your landscape? You have such big moving parts at all times. You can’t use last year’s priors because all these teams are completely different.
Mike: I like to use last year’s number, that’s the way I’ve always done it. I always look at last year’s stats for the first three or four weeks. I look to see what the number would have been on a game at the end of last year based on my model, and then make subjective adjustments. And for me, that’s what models or score projections have turned into. It’s really a starting point. I wouldn’t know what to do if I didn’t have that starting point. Because you take that score projection—and when I say models, I don’t just do one model. There’s several things I do, then maybe average them out, or weight them differently, to kind of get a weighted average number. I use that as my starting point. So, I’ll say at the end of last year for a particular game, the number would have been -10 and 56 total, and it’s -12 and 58 right now. And then I’ll ask myself, ‘Does that make sense based on coach and player changes or transfers or whatever?’ That’s kind of my process at the beginning of seasons. Then as data comes in throughout the year, I think you can get more comfortable using the current season’s stats.
Eddie: So I’m curious, how do you confidently adjust a team’s power rating throughout a season? You touched on the fact that you use the model as a beginning number. Let’s say by Week 3 that number is no longer realistic. How are you confidently moving numbers throughout the year?
Mike: I don’t do what you do where I’ll say Alabama will beat an average team by 28 points or whatever it is. That’s not really my process. I’m not doing manual adjustments every week. When you create a model, though, and you see it week to week … I know one year, the math I use really liked Kent State. This was before Kent State was tempo, flying all over the field, throwing every down. This was 10 years ago when they were horrible offensively. The math wanted me to bet them every week. I knew it was an anomaly with that team, and my model liked that team more than it should have for whatever reason. And I knew that in my head. So I knew that I couldn’t pay attention to that number, I had to look at some other power rating systems to be more accurate for a starting point for Kent State games. So that’s really what I’m doing more or less is, on a game by game basis, looking through, and if a number is way off, I’ll ask why it’s way off. Is it an anomaly, or does the model like this team for real? Or is there an injury? Or whatever.
Eddie: I write down every single box score, and I’m really big on strengths and weaknesses that I create throughout the offseason, and then in the middle of the season I’ll have more stuff to go with. But for instance, if I have a total on a team, I have a read on a team, or I think they’re going to be a really good over team because their pass defense isn’t up to par, or whatever. I don’t use just one week, but if I have two weeks of substantial evidence that this team is much better or much worse than I thought, then I have to make an adjustment. I use two weeks of data, two weeks of looking at my number, and then I’ll slowly start to move it. There are some instances where you have to move a team very quickly and sharply. I think everyone would agree that if you have a team that’s No. 11 in the country, and they don’t even look like a top-40 level team, you can’t just keep betting that team. You’ll go broke doing that. So I’m always conscious of what I think my power numbers should be, and how quickly or how fast I want to adjust it. That goes for sides and totals as well.
Mike: I think that’s been one of the big adjustments in the marketplace over the last five years. I think the bettors were much quicker to make an adjustment to a team like that in the past. It used to be, you find something, and you knew you had like three or four weeks you could bet it. In college basketball, maybe three or four games, and you’d expect to go 3-1 off one opinion. Nowadays, the books are making the same adjustment and they’re doing it quickly, and you might only have one shot at it.
Eddie: And I’d say, I think the books are overadjusting at times. So if you’re really confident in your number, you might have something there, you can play tic tac toe a little bit. But yeah, I agree with you 100 percent, the books have gotten much, much quicker to move their numbers, especially early in the season if they’re getting pounded by something. They’re not just going to sit there and take it like they used to.
Mike: Yeah, and I think that’s one of the things people should be looking for if you’re originating, is looking for those things where there’s a quick move that isn’t justified. A lot of times it is justified, but maybe in hoops, a team comes out and shoots 55 percent from 3-point range, and they shoot 60 threes the first three games, and they make 35 of them. That’s a high variance area in college basketball where you can potentially see something move too quick there, or a power rating adjustment that isn’t justified.
Eddie: I’ll give you an example of one that cost me money last year, and I think about this from time to time. But early in the season, you have anomaly games. Memphis and Arkansas State played a game that was like 59-55 last year. I had Arkansas State pegged as an over team, and I was neutral on Memphis. I moved my base total up on them (Memphis) after that game. And I think early in the season, if you move something, you’ve got to be pretty sure of it. This time, I did it just based off that one game. And Memphis never played another game like that the rest of the season I don’t think. I’m sure I lowered the number eventually and made adjustments, but the first couple of weeks, it probably cost me money. So you have to be careful early in the season with books adjusting, and you adjusting, just make sure you keep yourself in check is my best advice.