3

I have list of strings, I'm looking for lines like this:

Key: af12d9 Index: 0 Field 1: 1234 Field 2: 1234 Field 3: -10

after finding lines like this, I want to store each one as a dictionary {'key' : af12d9, 'index' : 0, 'field 1' : .... }, then store this dictionary to a list, so I will have a list of dictionaries.

I was able to get it working like this:

listconfig = []
for line in list_of_strings:
    matched = findall("(Key:[\s]*[0-9A-Fa-f]+[\s]*)|(Index:[\s]*[0-9]+[\s]*)|(Field 1:[\s]*[0-9]+[\s]*)|(Field 2:[\s]*[0-9]+[\s]*)|(Field 3:[\s]*[-+]?[0-9]+[\s]*)", line)
    if matched:
        listconfig += [dict(map(lambda pair: (pair[0].strip().lower(), pair[1].strip().lower()),
                                map(lambda line: line[0].split(':'),
                                    [filter(lambda x: x, group) for group in matched])))]

I'm just wondering if there could a better way (short and efficient) to do this because I think the findall will do 5 searches per string. (correct? since it returns a list of 5 tuples.)

Thank you.

Solution:

OK, with help of brandizzi, I have found THE answer to this question.

Solution: 

listconfig = []
for line in list_of_strings:
    matched = re.search(r"Key:[\s]*(?P<key>[0-9A-Fa-f]+)[\s]*" \ 
                        r"(Index:[\s]*(?P<index>[0-9]+)[\s]*)?" \ 
                        r"(Field 1:[\s]*(?P<field_1>[0-9]+)[\s]*)?" \ 
                        r"(Field 2:[\s]*(?P<field_2>[0-9 A-Za-z]+)[\s]*)?" \ 
                        r"(Field 3:[\s]*(?P<field_3>[-+]?[0-9]+)[\s]*)?", line) 
    if matched:
        print matched.groupdict()
        listconfig.append(matched.groupdict())
Improve this question
7
  • Is "Ring" part of "Field 3" or part of "1234"? Commented Apr 13, 2011 at 17:27
  • Do you really have "Field 1", "Field 2", ... with blanks? That's strange format. Field1, Field2, ... would make it much simpler. Are you free to choose or are the blanks required? Commented Apr 13, 2011 at 17:29
  • If the keys are hex numbers, you probably want [0-9A-Fa-f] Commented Apr 13, 2011 at 17:31
  • if you look at his regex you'll see that the ':' delineates the key from the value and a ' ' delineates the value from the following key. Commented Apr 13, 2011 at 17:36
  • 2
    @sudo If there are always 'Key' , 'Index', 'Field1', 'Field2', 'Field3' in each string element of the list, it's clumsy to encumber the data structure with them: ('af12d9', '0', '1234', '1234', '-10') is sufficient , you know that the second element is Index and the last is Field3. - Also: listconfig += is a bad practice because it creates a new list and assign the name listconfig to it. Use append() instead. Commented Apr 13, 2011 at 17:59

6 Answers 6

Reset to default
5

Firstly, your regex seems to not work properly. The Key field should have values which could include f, right? So its group should not be ([0-9A-Ea-e]+) but instead ([0-9A-Fa-f]+). Also, it is a good - actually, a wonderful - practice to prefix the regex string with r when dealing with regexes because it avoids problems with \ escaping characters. (If you do not understand why to do it, look at raw strings)

Now, my approach to the problem. First, I would create a regex without pipes:

>>> regex = r"(Key):[\s]*([0-9A-Fa-f]+)[\s]*" \
...     r"(Index):[\s]*([0-9]+)[\s]*" \
...     r"(Field 1):[\s]*([0-9]+)[\s]*" \
...     r"(Field 2):[\s]*([0-9 A-Za-z]+)[\s]*" \
...     r"(Field 3):[\s]*([-+]?[0-9]+)[\s]*"

With this change, the findall() will return only one tuple of found groups for an entire line. In this tuple, each key is followed by its value:

>>> re.findall(regex, line)
[('Key', 'af12d9', 'Index', '0', 'Field 1', '1234', 'Field 2', '1234 Ring ', 'Field 3', '-10')]

So I get the tuple...

>>> found = re.findall(regex, line)[0]
>>> found
('Key', 'af12d9', 'Index', '0', 'Field 1', '1234', 'Field 2', '1234 Ring ', 'Field 3', '-10')

...and using slices I get only the keys...

>>> found[::2]
('Key', 'Index', 'Field 1', 'Field 2', 'Field 3')

...and also only the values:

>>> found[1::2]
('af12d9', '0', '1234', '1234 Ring ', '-10')

Then I create a list of tuples containing the key and its corresponding value with zip() function:

>>> zip(found[::2], found[1::2])
[('Key', 'af12d9'), ('Index', '0'), ('Field 1', '1234'), ('Field 2', '1234 Ring '), ('Field 3', '-10')]

The gran finale is to pass the list of tuples to the dict() constructor:

>>> dict(zip(found[::2], found[1::2]))
{'Field 3': '-10', 'Index': '0', 'Field 1': '1234', 'Key': 'af12d9', 'Field 2': '1234 Ring '}

I find this solution the best, but it is indeed a subjective question in some sense. HTH anyway :)

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

@brandizzi Ha, thanks a lot. This really helps. I found out that you simply just use re.search, and get the dictionary using groupdict(), here is my improved version: matched = re.search(r"Key:[\s]*(?P<key>[0-9A-Fa-f]+)[\s]*" \ r"(Index:[\s]*(?P<index>[0-9]+)[\s]*)?" \ r"(Dose:[\s]*(?P<dose>[0-9]+)[\s]*)?" \ r"(Energy:[\s]*(?P<energy>[0-9 A-Za-z]+)[\s]*)?" \ r"(Ring Intensity:[\s]*(?P<intensity>[-+]?[0-9]+)[\s]*)?", line) if matched: print matched.groupdict()
@sudo you just found the solution. You really should post it as an answer to your own question and then mark it as the correct answer! It is more necessary yet because you can format the code in a good way in an answer but cannot do it as a comment. Seriously, post your answer as the correct one :)
But is there a way to handle the mixed ordering? e.g. Index comes before Key?
@sudo dicts are not ordered by default. Python 2.7 provides an OrderedDict class in the collections module - if you need to preserve the order and can use Python 2.7, you can use my solution using OrderedDict(...) instead of dict(...). However, do you really need it? You already know the order since you built the regex and can just use the results in the correct order. Also, if the order is important and the lines have always the same structure, maybe the tip of eyquem in the comment of your question can be the best solution. Are these solutions good for you?
@brandizzi The solution was good enough for what I was doing. But I was just curious about a different senario. Thanks for all your suggestion.
1

OK, with help of brandizzi, I have found THE answer to this question.

Solution: 

listconfig = []
for line in list_of_strings:
    matched = re.search(r"Key:[\s]*(?P<key>[0-9A-Fa-f]+)[\s]*" \ 
                        r"(Index:[\s]*(?P<index>[0-9]+)[\s]*)?" \ 
                        r"(Field 1:[\s]*(?P<field_1>[0-9]+)[\s]*)?" \ 
                        r"(Field 2:[\s]*(?P<field_2>[0-9 A-Za-z]+)[\s]*)?" \ 
                        r"(Field 3:[\s]*(?P<field_3>[-+]?[0-9]+)[\s]*)?", line) 
    if matched:
        print matched.groupdict()
        listconfig.append(matched.groupdict())
Improve this answer

Comments

0 
import re

str_list = "Key: af12d9 Index: 0 Field 1: 1234 Field 2: 1234 Ring Field 3: -10"
results = {}
for match in re.findall("(.*?):\ (.*?)\ ", str_list+' '):
    results[match[0]] = match[1]
Improve this answer

1 Comment

note: this won't check to make sure you've got valid data as Mark Tozzi said -- you'd need some sort of bounds checking to verify it's accurate, but based on the rules you've got listed, this is a MUCH easier and faster way to do this.
0

The pattern in your example is probably not matching your example data due to the "Ring". Here is some code which might help:

import re
# the keys to look for
keys = ['Key','Index','Field 1','Field 2','Field 3']
# a pattern for those keys in exact order
pattern = ''.join(["(%s):(.*)" % key for key in keys])
# sample data
data = "Key: af12d9 Index: 0 Field 1: 1234 Field 2: 1234 Ring Field 3: -10"
# look for the pattern
hit = re.match(pattern,data)
if hit:
    # get the matched elements
    groups = hit.groups()
    # group them in pairs and create a dict
    d = dict(zip(groups[::2], groups[1::2]))
    # print result
    print d
Improve this answer

Comments

0

You could use a parser library. I know Lepl, so will use that, but because it is implemented in Python it will not be so efficient. However, the solution is fairly short and, I hope, very easy to understand:

def parser():
  key = (Drop("Key:") & Regexp("[0-9a-fA-F]+")) > 'key'
  index = (Drop("Index:") & Integer()) > 'index'
  def Field(n):
      return (Drop("Field" + str(n)) & Integer()) > 'field'+str(n)
  with DroppedSpaces():
      line = (key & index & Field(1) & Field(2) & Field(3)) >> make_dict
      return line[:]
p = parser()
print(p.parse_file(...))

It should also be relatively simple to handle a variable number of fields.

Note that the above is not tested (I need to get to work), but should be about right. In particular, it should return a list of dictionaries, as required.

Improve this answer

Comments

0

Your solution would perform better if you did this[*]:

import re

from itertools import imap

regex = re.compile(flags=re.VERBOSE, pattern=r"""
    Key:\s*(?P<key>[0-9A-Fa-f]+)\s*
    Index:\s*(?P<index>[0-9]+)\s*
    Field\s+1:\s*(?P<field_1>[0-9]+)\s*
    Field\s+2:\s*(?P<field_2>[0-9A-Za-z]+)\s*
    Field\s+3:\s*(?P<field_3>[-+]?[0-9]+)\s*
""")

list_of_strings = [
    'Key: af12d9 Index: 0 Field 1: 1234 Field 2: 1234 Field 3: -10',
    'hey joe!',
    ''
]

listconfig = [
    match.groupdict() for match in imap(regex.search, list_of_strings) if match
]

Also, it'd be more succinct. Also, I fixed your broken regex pattern.

BTW, the result of the above would be:

[{'index': '0', 'field_2': '1234', 'field_3': '-10', 'key': 'af12d9', 'field_1': '1234'}]

[*] Actually - no, it wouldn't. I timeit'ed both and neither is faster than the other. Still, I like mine better.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.