Prepositions in front of an infinitive are not related to the infinitive verb itself, but they are needed to specify the type of information conveyed by that non-main (i.e., not the verb nor the subject) part of the phrase.
In your examples:
- "ho deciso di portare" -> is split as "ho deciso" (I decided) and the verb "decidere" needs a part-of-the-phrase with the preposition "di" to convey the information of what was decided, if it was an action.
- "vado a prendere" -> verbs of motion typically require the preposition "a" in parts-of-the-phrase that tell where somebody moved (if the location does not contain. In that case, the preposition would be "in"). You can find it also with non-verb destinations: "vado a casa", "vado al mare".
- "voglio chiedere" -> as is typical in many languages, direct objects do not need a preposition. What do you want? I want to ask (voglio chiedere). In this situation, "chiedere" (to ask) is direct object, so it does not need a preposition.
Edit: TL;DR answer is yes, you have to memorize them, but they don't come from the verb you are putting in the infinitive mode, but from the ones that are used for the part-of-phrase you are putting the infinitive verb into.
Edit #2: Adding a link that may be useful to the OP (http://www.zanichellibenvenuti.it/wordpress/?p=4401) it contains a list of main verbs and the required preposition for the infinitive verb associated. There are 4 main groups: verbs not requiring prepositions before an infinitive, verbs requiring "A" (and a "figurative static place" as part-of-phrase, which is why in some verbal situation the preposition may be replaced with "NEL"), verbs requiring "DI" (and a "specification" part-of-phrase), and verbs supporting both "DI" and "PER".