Does Antlr parse multiple tokens from a given text?

[AsfarHorani]

Given text: I live in Paris but was born in Berlin.
LexerRule:
City: 'Berlin' | 'Paris';
Grammar: City;

Upon running I am only getting Paris and not berlin.

Please help

[KvanTTT]

It's not possible but it's possible to use a parser rule instead:

city: Berlin | Paris;
Berlin: 'Berlin';
Paris: 'Paris';

[AsfarHorani]

@KvanTTT These two are just an example I want to use thousands of cities would it not be complicated to manage them like this? might also increase memory usage

Why parser rule cant be like?
Grammar:
city: city

//lexer rule
city: 'berlin' | 'paris' | 'london' .... <1000 values>

sample text: "Top cities of europe are London, Paris, Berlin etc. All have huge population. We can also consider madrid and rome with over 2 Mil population"

I am expecting to get all cities mentioned

[jimidle]

I don't think that ANTLR is the right tool for this task. Just call GPT API. It seems like you are looking to parser freeform text, not a definite structure. You could also use something like spacy in Python.

[AsfarHorani]

You guessed it right nut why Antlr is not a right tool for that? It is support to parse grammar and patterns. Now Imagine a address pattern like 123 Street abc road, 2098 New york, USA cant we create grammars to parse them?

[p7r0x7]

Address patterns can be mapped to an ANTLR grammar, yes.