2
\$\begingroup\$

I want to return a NumPy array of Element IDs with at least one Node being in the interesting Nodes list.

I have:

  • A list of interesting Nodes, Interesting_Nodes.
    The size is ~300,000 items.

  • A list of sublists which represent Node forming an Element, Nodes.
    The size is ~1,000,000 lists of ~20 values.

    Nodes[i] contain the 20 Nodes creating Elements[i]. Which is an integer ID.

  • A list of Elements, Elements. The size is ~1,000,000 items.

Here is an example of these lists.

import numpy as np
Interesting_Nodes=[1,2,10,40,400,1000]
Elements=[1,2,3]
Nodes=[[1,20,25],[30,400,35],[500,501,502]]

In this case the function will return [1,2] because Elements[0] have the Node 1 in the Interesting_Nodes list and Elements[1] contains the Node 400 which is also in the Interesting_Nodes list

I wrote this it seems to work but is very slow. Is there a way to perform this faster?

def recuperation_liste_element_dans_domaine_interet(Interesting_Nodes,Elements,Nodes):
    Liste_elements=list([])
    for n in Interesting_Nodes:
        id=np.where(Nodes==n)
        Liste_elements=Liste_elements+list(Elements[id[0]])
    nbrListe = list(set(Liste_elements)) # remove all the element duplication
    return np.array(nbrListe)

Another way to perform that I think could be: (be still too slow)

def recuperation_liste_element_dans_domaine_interet(Interesting_Nodes,Elements,Nodes):
    Liste_elements=[0]*len(Elements)
    compteur=0
    j=0
    for n in Nodes:
        if any(i in Interesting_Nodes for i in n):
            Liste_elements[compteur]=Elements[j]
            compteur=compteur+1
        j=j+1
    Liste_elements=Liste_elements[:compteur]  
    return np.array(Liste_elements)
\$\endgroup\$
0

1 Answer 1

Reset to default
3
\$\begingroup\$

I'm assuming Elements is always a list of continuous indices from zero to the length of Nodes minus one and thus strictly not needed.

As a general guideline, explicit loops tend to be slow and also hard to read (and non-Pythonic).

To solve your problem efficiently, use data structures (namely sets):

import numpy as np

# Suppose you start with these lists
u = [1,2,10,40,400,1000]
n = [[1,20,25],[30,400,35],[500,501,502]]

# Convert them into sets
u = set(u)
n = [set(x) for x in n]

#  An individual bit is true if the corresponding list is "interesting"
idx = [bool(a.intersection(u)) for a in n]

# Perhaps convert to integer indices if you wish
np.where(idx)
\$\endgroup\$
4
  • \$\begingroup\$ thanks a lot for your help. I figured out myself that set is much more powerful to do what I expected 1 hour ago ;) But not as "clean" as you write. I think it would do the job perfectly ! Thanks again ! \$\endgroup\$ Commented Oct 2, 2020 at 18:27
  • \$\begingroup\$ By the way in fact i need Elements because it is not a continuous list. It coub be [10,1000,300]. But with idx I suppose I can juste write Elements[idx] to have my answer. \$\endgroup\$ Commented Oct 2, 2020 at 18:45
  • \$\begingroup\$ Did you see an improvement in your "very slow" execution time? What kind of improvement did you see? \$\endgroup\$ Commented Oct 3, 2020 at 13:29
  • 1
    \$\begingroup\$ a HUGE improvement !!! With the previous version, with the "utltimate" amount of data, i never manage to reach the end. With the new version, the code finish in few minutes. Thank you again @Juho \$\endgroup\$ Commented Oct 4, 2020 at 16:56

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.