Problem causes and solutions
Before you will continue, please be sure that aperture is working on your lens. In case that aperture cable is broken and aperture is stuck on some small value then not enough light will go to phase AF sensor and autofocus will not work too. If this is your case, inspect and replace aperture flex cable if needed.
According to problems behavior I sorted causes into two categories:
1. Focal length brush
AF is working properly only on some focal lengths. There are focal lengths where focus is hunting and can't focus. Situation looks like when you are trying to use macro lens with focus limit enabled and shooting portrait photo. Lens is focusing but in bad range, so focus cannot be achieved. Personally I don't observed such problem, but if lens is getting focal length information to adjust focus range for faster AF, then you can imagine what will happen when lens get bad focal length information. Yes you are right, it is just trying to focus in bad range and that is impossible. Focal length is obtained from linear encoder which is consisting from flexible printed circuit which is fastened on lens barrel and precise brush. Flexible cable can be broken because it bending every time you zoom in and out, or gold contacts on cable and brush could be damaged, dirty, etc...
Solution is easy. First access brush, it is hidden under rubber belt grip on zoom ring. Under this belt is sticker which is covering hole for brush. remove sticker, unscrew brush and inspect it. Be careful because brush is very fine and can be easily damaged by even small force. If you will eventually bend it, don't panic, get precise tweezer, magnifier glass and bend it gently to original geometry. After you removed brush, clean contacts under it (you have to zoom in and out to access full length if tracks) with 100% alcohol. Of course clean also brush. After cleaning you can apply silicone grease, apply just a tiny layer on tracks to avoid excessive wearing of tracks and brush. Too much grease could cause bad contact between brush and tracks or can even taint inside barrels of lens. Now put all together and test it. If is everything ok you are done.
1. AF position sensor
Because Canon is using USM motors which are not using gearbox but rotor of this motor is directly focusing barrel (which is great but...) then need to have very precise and tiny position sensor. This sensor looks in standard lens like wheel with pattern holes which are interrupting light and thus gives pulses to processor. Camera then send command, focus xxx steps to some direction, and lens just move motor until xx pulses are get from that position sensor. That's all the magic. Because there is gearbox, motor will rotate many timer to make tiny movement of focus ring, so pattern wheel can be relatively large but Canon is not using gearbox on some lenses so this type of patter wheel is useless here. What canon implement is using same principe, but have much larger resolution and thus much smaller size. They are using two transparent plastic tracks with triangular micro lenses which are one on another, when these two tracks are moving light from LED which is going through them is interrupted (just like in previous case). Problem is that these lenses (or call it whatever you like) on plastic tracks are so tiny and two tracks are so close one to another that any hard particle which will go between them will scratch surface and after few moves is track damaged. There will be deep grooves, on rest of track will be more and more particles from crushed plastic and micro lenses are unable to interrupt light. No interrupted light, no pulses to processor. Camera want to move focus on few steps, motor is started, but no pulses are get from position sensor. So motor is moving forward and backward without any result. Of course if only part of track is damaged, then AF is running properly in this focus range, obviously.
Solution in early stage is easy, but early stage means that you will do these steps immediately as you are feeling that AF is starting to run improperly. In later stages is solution easy too, but much more expensive.
Let's suppose that you are thoughtful owner of your photo gear, you feel that AF starts hunting and focusing takes longer than usual. Disassembly lens and just clean incremental sensor with 100% alcohol. Inspect tracks under microscope for grooves or damage. If you see damage which will cause large diffusion of light, then use polish liquid for plastic materials (must be fine without abrasives) and polish these areas. Put all together and test lens, should be working fine now. For pictures with description please see below.
Now sad part of story, if you ignored fact that AF is not as good as new, and now it don't run at all. You can of course try procedure above, if you are lucky it will work because there could be just a lot of dust from scratched plastic tracks and after cleaning there will be still sufficient lens relief for light refraction and thus sensor will work. If not you have to replace incremental sensor tracks. Don't worry there are for 50 USD on eBay of aliexpress, look for "Canon Lens Strip Sensor Bar" or something like that.
If track if ok but you lens is hunting focus then flex cable from this sensor is broken, or LED transmitter or receiver is broken. These can be replaced too, LED transmitter and receiver are in element14 or farnell available for few $ but I didn't searched them yet so can't tell you part number for compatible part. Flex cable can be inspected and measured if all track are conducting. LED transmitter and receiver can be tested by multimeter and oscilloscope with IR sensor probe. Diagnostic precision depends on your equipment.
|Disassembled focus ring. On photo is USM motor (on bottom) and incremental sensor (on top). Remove two screws and gently remove slider and transparent track.|
|Incremental sensor track with pattern and slider with pattern.|
|You can see deep grooves and in that dim track are under microscope many micro lenses. Nice macro photo of this track is here https://www.flickr.com/photos/nivek_woods/sets/72157630430286912/|
|Use baby cleaning sticks with polish liquid for plastic materials and polish track and slider. Start with few strokes in same direction as grooves.Just 5-10 strokes. Then primary polish track and slider in same direction as red lines on photo. Take stroke up, down and move 1 mm to side. Do this on whole damaged area 5-10 times. Inspect track during procedure under microscope for changes, so you can control your steps to better results.|
Feel free to contact me with any questions or improvements. I will try to add more photos here but at moment I pushed at least most important information here.