tokenize
Hello, you have come here looking for the meaning of the word tokenize. In DICTIOUS you will not only get to know all the dictionary meanings for the word tokenize, but we will also tell you about its etymology, its characteristics and you will know how to say tokenize in singular and plural. Everything you need to know about the word tokenize you have here. The definition of the word tokenize will help you to be more precise and correct when speaking or writing your texts. Knowing the definition oftokenize, as well as those of other words, enriches your vocabulary and provides you with more and better linguistic resources.
English
Etymology
Pronunciation
- (General American) IPA(key): /ˈtoʊ.kən.aɪz/
Verb
tokenize (third-person singular simple present tokenizes, present participle tokenizing, simple past and past participle tokenized)
- (transitive, computing) To reduce to a token or set of tokens by lexical analysis.
- (transitive, computing) To substitute sensitive data with meaningless placeholders.
- (transitive) To treat as a token minority.